Recent gravitational wave observations allow us to probe gravity in the strong and dynamical field regime. In this work, we focus on testing Einstein-dilation Gauss-Bonnet gravity which is motivated by string theory. In particular, we use two new neutron star black hole binaries (GW200105 and GW200115). Adopting the leading post-Newtonian correction and carrying out a Bayesian Markov-chain Monte Carlo analysis, we derive the 90% credible upper bound on the coupling constant of the theory. This bound is stronger than the bound obtained in previous literature by combining selected binary black hole events in GWTC-1 and GWTC-2 catalogs. In order to check the validity of the effect of higher post-Newtonian terms, we derive corrections to the waveform phase up to second post-Newtonian order by mapping results in scalar-tensor theories to Einstein-dilation Gauss-Bonnet gravity. We find that such higher-order terms improve the bounds by about 10%.