If you have another employment where you are already paid over the upper threshold for NIC, then your employer is using the wrong NI table letter, but he has no choice, as you haven't supplied the necessary certification. (PAYE administration is evidence based)
Once you supply your employer with the certificate, then they will replace your NI letter with J rather than A.
Direct them to booklet CA38, if you have problems after supplying the certificate.
If your two employments combined send you over the upper threshold, then you may still be able to recover overpaid NIC, but that would be under a separate process.