Hmm went an changed it and I definitely had to unbolt the slave cylinder and move it out of the way. It sandwiches a plastic chain guard (which I think is just there to catch goop coming off the chain). I did unbolt it with the line still attached, I got some fluid weeping from it too not a lot but I tried to minimize the time it was off. I tried to depress the plunger in with a large locking plier which I think was unnecessary and may have bent the plunger.
But I have it all back together now and the clutch seems ok. There was some fluid weeping from it and I cleaned it a buncha times and took it out for a ride and it shifts fine. will periodically check the slave cylinder to see if it's still weeping for now, hope I didn't damage it.
But to answer my question it's totally fine to just unbolt the slave cylinder.