Schrock85 You are absolutely right.
Removing the background is one thing, quite easy.
The hardest part is how to place your segmented actor into a new background keeping lights, colors and all this stuff natural.
The problem is known as Image Matting
https://paperswithcode.com/task/image-matting#datasets
For now, we are trying a naive approach: segment actors -> replace the background, done. This is mostly to make an MVP and collect data.
Later we will probably try to spend some more time on this to make it a bit more natural by solving the Image Matting problemπ