Fussiness at the breast

I have a question. My baby is 3 weeks old and until a few days ago has been
nursing beautifully. In the past couple of days, he has been struggling and
fussing after a few seconds of nursing. I have heard that babies can get
frustrated that milk is coming to quickly so I've tried taking him off and
pumping and putting him back on and he still follows the same pattern of
sucking/fussing. I have been trying to work on pumping my milk and
introducing a bottle occasionally with him since I am absent from him for a
couple hours during the day. Could this be causing too much milk to come in
and causing him to do this pattern? He does the same thing on the bottle as
well. Also, he seems to do it more in the evening and usually does fine
during the day. I have been burping him regularly as well to see if it's gas
and he'll burp but still do the same thing once he's on the breast again.
Help! We are both often ending in tears these days...