Let $G$ be a finite group. A subgroup $H$ of $G$ is called an $mathcal{H}$-subgroup in $G$ if $N_G(H)cap H^{g}leq H$ for all $gin G$. A subgroup $H$ of $G$ is called a weakly $mathcal{H}^{ast}$-subgroup in $G$ if there exists a subgroup $K$ of $G$ such that $G=HK$ and $Hcap K$ is an $mathcal{H}$-subgroup in $G$. We investigate the structure of the finite group $G$ under the assumption that every cyclic subgroup of $G$ of prime order $p$ or of order $4$ (if $p=2$) is a weakly $mathcal{H}^{ast}$-subgroup in $G$. Our results improve and extend a series of recent results in the literature.