Toss the pellets in the garbage. Get a tin of loose black powder FF for your 54 load it with 85 to 90 grains and your gun will fire perfect. Don't forget the need to clear the nipple and pan with a nipple pick though before putting on your #11 primer. ( a safety pin works good ).
once you get onto loose powder and have your preloads made it's faster than pellets and much much cheaper. You also have the ability to load exactly what shoots best out of a rifle.
I quit using pellets in my TC omega about 6 years back for the Yukon gold Jim shockeys, it works awesome I've fired hundreds of rounds never had a misfire, left the ml loaded ( not primed ) for months at a time stuff still goes off flawlessly and cleanup with that powder isn't required immediately after firing.. ( very handy during hunting season )
but most importantly if you load 125 grains of that powder with a 250 grain shockwave sabot you have a 200 yard tack driver.
