It is a no-brainer to use the collected data points instead of user feedback in this case. Only a small set of users would down vote enough to make it relevant. Also hard to measure the quality of the votes and how they age.
Well, no surprise this was down voted since most people in this thread never seems to have worked in this field. I have worked with recommendations on a service with millions of users. We do not guess. We look at things like conversion rates to tell us what works. If one think down votes would play a significant part in training a model you simple does not understand the amount of metrics a modern service collects and what impact different variables have.