Let $G$ be a finite $p$-group and $N$ be a normal subgroup of $G$ with $|N|=p^n$ and $|G/N|=p^m$. A result of Ellis (1998) shows that the order of the Schur multiplier of such a pair $(G,N)$ of finite $p$-groups is bounded by $ p^{\frac{1}{2}n(2m+n-1)}$ and hence it is equal to $ p^{\frac{1}{2}n(2m+n-1)-t}$ for some non-negative integer $t$. Recently, the authors have characterized the structure of $(G,N)$ when $N$ has a complement in $G$ and $t\leq 3$. This paper is devoted to classification of pairs $(G,N)$ when $N$ has a normal complement in $G$ and $t=4,5$.