Hermans survived four climbs to take the final sprint as he pulled away in the final metres to take a convincing win over Rui Costa (UAE Abu Dhabi) and Jakob Fuglsang (Astana).
The second stage offered not only beautiful landscapes, but also demanding climbs that Oman is known for.
After the battle between the sprinters in the Tuesday's opening stage, it was time for the general classification (overall lead) contenders to show how good their legs were.
Part of the leading group on the final climb of the day, Hermans awaited the ultimate moments to power to his first ever stage success in Oman and capture the overall lead.
‘Had a free role’
Hermans took the leader's red jersey off the back of Alexander Kristoff (Katusha-Alpecin), the winner of stage one.
Hermans was supposed to be helping his BMC team leader and compatriot Greg Van Avermaet, the Olympic champion, but jumped on his opportunity to claim victory.
"Greg was supposed to contest the sprint finish, whereas me, I had a free role, in particular marking Romain Bardet. I tried my luck at the best moment 150m from the [finish] line," said Hermans.
Van Abermaet finished ninth with Bardet, second behind Chris Froome at last year's Tour de France, was tenth.
It took a while for the day’s break group to form and several large groups tried to get away, but were brought back.
The field was flying along, at more than 50kmph for the first hour. Then after about 45km, Mark Christian (Aqua Blue Sport) and Preben Van Hecke (Sport Vlaanderen-Baloise) got away.
The field was happy to let them go and the gap rapidly hovered near the eight-minute mark.
The last three climbs came in the final third of the stage, and as the first of those appeared, the gap started coming down. With 40km and two more climbs, it was down to three minutes and decreasing with regularity.
Van Hecke was caught with just over 20km to go, and Christian fought on alone with a handful of seconds before succumbing to the peloton.
Best young rider Lasse Norman Hansen (Aqua Blue) attacked on the lead-up to the final climb, soon to be joined by Daniel Eaton (UnitedHealthcare).
They carried on for 35 seconds up the fearsome Al Jissah, but were gathered in with only 2km to go.
A reduced and splintered peloton approached the finish line as Hermans moved away to take an easy win, with Costa and Fuglsang, his nearest chasers, fighting it out for the remaining podium positions.
Dimension Data’s Merhawi Kudus was fourth. Thursday's 162km third stage will begin at Sultan Qaboos University and finish at Quriyat.
Results [top five]: Stage two : 1 Ben Hermans (BMC Racing Team), 2 Rui Costa (Team UAE Abu Dhabi), 3 Jakob Fuglsang (Astana Pro Team), 4 Merhawi Kudus (Dimension Data), 5 Nathan Haas (Dimension Data).
General classification: 1 Hermans , 2 Costa, 3 Fuglsang, 4 Kudus, 5 Haas.