The load effect of Rx filter on Tx filter in Bulk acoustic wave (BAW) duplexer can bring about the degradation of the insertion loss performance and can cause performance surplus of the band rejection of Tx filter. In order to solve this problem, a design optimization method of BAW duplexer was proposed in this work. The resonance area of series FBAR and ratio of resonance area of parallel FBAR to series FBAR were set up as two groups of optimization variables. By sacrificing the excess out band rejection of Tx filter, the gradient base optimization algorithm was used to obtain the final values of the two groups of the optimization variables. The application progress of the proposed method was showed by an optimized design example of BAW deplexer in FDDLTE band 7. The simulation result showed that the insertion loss of Tx filter was improved from 2 dB to 1.1 dB, and the ripple in band was improved from 1 dB to 0.3 dB, which verifies the feasility of this method.