Deserter's Gold Quest bug?

[Goranovic]

Hey, I have finished all the '?' markers and explored the whole White Orchard area.
Now when I start the named quest above (Deserter's Gold) I come at the mill, cleared all the deserters out without a problem and head down the hole for the chests?

But they don't light up with Witcher Sense nor can I open them with the Temerian Key I found before? When I approach the chests a button pops up (A on Xbox controller) to unlock it. Sad enough it doesn't work.

Anyone else having this issue? I haven't completed the quest yet since it's still under 'Treasure Hunt' and neither am I getting the usual quest marker on the minimap.