An intrusion detection system utilizes various statistical information scattered around within the system. It can abstract information from the normal behaviors of a system and detect attacks regardless of whether or not the system has observed them before. In this paper, we propose an effective IDS that improves the modeling time and performance with only considering the events of privilege flows based on the domain knowledge of attacks. Proposed privilege change model is evaluated with fixed sequences from BSM data on the situation where transitions between UID and EUID occur. Analysis of HMM experiment shows that using privilege change data is approximately 250 times faster and produces 132 times decrease in quantity of sequences than using all data.