Aiming at solving the problem that the calibration accuracy of inertial measurement unit (IMU) will be affected if it is interrupted by the ground vibration when it is self calibrated in outfield, the influence of gross error on the estimated value of Kalman Filtering (KF) is analyzed firstly, then a modified KF based on Robust Estimation is proposed to be used in the data processing of IMU self calibration in outfield. By descending weights of abnormal data continuously via equivalent weight function, the influence of abnormal data on IMU outputs is minimized. Taking the advantages of KF and equivalent weight function, the new method is not only real time but also robust. Experiment results show that comparing with Sage Husa adaptive KF and Fault Tolerant KF, the robust KF is more robust, its rapidity of convergence is faster. The accuracy of single testing is improved at least one order. It can degrade the influence of abnormal data on the calibration accuracy effectively.