In short, DPO is not better than PPO. This is because DPO is derived from so called BT reward assumption that pairwise data preference is collected. Through mathematical formulations, you can learn the preference and the action at the same time. However, PPO and other on-policy (training samples are strictly generated by the LLM) doesn't need such assumption. For example, in coding and math problems it is possible to get binary reward. Many research shows DPO is ok if you don't take much care on OOD performance.
I can understand the concerns of the technology might be abused in DoD but I should point out that it is much better to see this result on Science Robotics, TRO, ICRA, and IROS etc. Military is behind many technology innovations but this doesn't mean all technologies should be used for military purpose only.
Agree, aerial vehicles are slightly better than maritime vehicles in this sense. I left maritime robotics because I couldn’t find any decent paying job that is not defense related :( What we can do as a student team is to open-source it.
This is a hybrid Remote Operated Vehicles (ROVs) and Autonomous Underwater Vehicles (AUVs). The vehicle can be controlled using a joystick(^1) using a wired connection while operating as a ROVs.
Every year we upgrade our autonomous sailboat controller to the latest Raspberry Pi. This year we face a situation to choose from Jetson Nano and RPi 4. Even the decision is hard to made, now is an exciting moment for robot makers.
Maritime and airplane emission is definite an issue after the grid power transferred to renewables. The problem is the low energy density and the variance in renewable resources.
I guess for transportation it is more practical to aim for net carbon neutral.
Reminds me 锟斤拷 due to Unicode replacement character misinterpretation problem. When placeholder 'U+FFFD' decoded using GBK it will displayed as these characters. Some of glitches can still be found online, e.g.,
https://docs.oracle.com/cd/E19199-01/817-4244-10/preface.htm...
