Ultrasound imaging has developed into an indispensable imaging technology in medical diagnosis and treatment applications due to its unique advantages, such as safety, affordability, and convenience. With the development of data information acquisition technology, ultrasound imaging is increasingly susceptible to speckle noise, which leads to defects, such as low resolution, poor contrast, spots, and shadows, which affect the accuracy of physician analysis and diagnosis. To solve this problem, we proposed a frequency division denoising algorithm combining transform domain and spatial domain. First, the ultrasound image was decomposed into a series of sub-modal images using 2D variational mode decomposition (2D-VMD), and adaptively determined 2D-VMD parameter K value based on visual information fidelity (VIF) criterion. Then, an anisotropic diffusion filter was used to denoise low-frequency sub-modal images, and a 3D block matching algorithm (BM3D) was used to reduce noise for high-frequency images with high noise. Finally, each sub-modal image was reconstructed after processing to obtain the denoised ultrasound image. In the comparative experiments of synthetic, simulation, and real images, the performance of this method was quantitatively evaluated. Various results show that the ability of this algorithm in denoising and maintaining structural details is significantly better than that of other algorithms.