A novel design of ultrawideband (UWB) filter is proposed aiming to realize high stopband rejection and take into account the miniaturization. Firstly, with separate electric and magnetic coupling path (SEMCP) topology, two transmission zeros (TZs) are produced above and below the passband which can be independently controlled by simply tuning dimensions of the floating metal patch and spiral defect ground structure (DGS), thus achieving high stopband attention are facilitated the miniaturization of filter. Then, by using substrate integrated waveguide (SIW) with periodical DGS, better stopband performance is achieved while further improving the frequency selectivity. Finally, a filter works at 8.1 GHz with bandwidth of 67% is designed and fabricated. The simulation results are in good agreement with the test results, which proves the reliability of the design method.