Absolutely. This is for secondary houses mostly though, it's slightly easier to expel those from your main accomodation.

But I think in any case your ownership of the property is not challenged. It's "just" that you cannot use it, but eventually you can get it back.