petermc I think it's doable, but it's not nearly as convenient to manage the Watch Later list as it is the hearts, because you have to click on the submenu for every video when browsing to check whether it's on the list or not, and your focus has to switch from the submenu button to the Watch Later status in the menu every time, which is awkward. When using hearts, you don't ever have to context switch; whether something is in your collection or not is shown right with the video listing.
Having a visible icon for videos marked as Watch Later would help save on the video-by-video checking, and even better if it's clickable so you don't have to go to the submenu at all. Frankly I think if it's not as easy to use as the hearts, then most people will still gravitate towards using those to track their collection first.