We present a method for tracking the 3-axis orientation of a monocular camera using orthogonal vanishing points detected in individual frames of a sequence of images. Robust and real-time vanishing point detection is done using a standard line segment detection method and an adaptive RANSAC algorithm. Vanishing points and corresponding vanishing directions found in consecutive frames are associated with each other to produce a sequence of orientation quaternions, which is processed by an extended Kalman filter. Experiments with a consumer-level, handheld mobile device indicate that the accuracy of the proposed method is comparable with those of consumer-grade inertial motion sensors.