Track building as a mask

Hi.

i would like to know if its possible to track a building and use it as a mask in 3d space ?

The project is an artshow in a soon to be demolitioned five story building.
What im trying to do is that there would be animated 3d objects floating around the building and they could go hidden behind the building and appear once they revoled around it and people waiting in queue would be able to view them.

Can there be multiple image trackers or can this be done only in gps tracking or something?
If a person is near the building and person pans up to view the top of the building will the tracker be able to track from perspective distorted data?

if this can be done can someone link me to the correct path to how to do this since im super noob with this.

Thanks in advance.