HSL adjustment layer effects all layers beneath it ...

Turn it into a clipping mask to only effect the layer beneath

[Under Layers ...]

Note – fill layers are nice ways to get fast broad effects.