The home side then went to stumps on 37-1 with KL Rahul and Washington Sundar still at the crease, still a deficit of 122 on South Africa's total.
The Proteas were dealt a blow when captain Temba Bavuma announced at the toss that fast bowler Kagiso Rabada failed a late fitness test due to a rib injury and will miss the game.
Bavuma would have been happier after calling correctly and opting to bat, then watching opening pair Aiden Markram and Ryan Rickelton put on a 57-run stand.
It was Bumrah that made a double breakthrough, first bowling Rickelton for 23 and then having Markram caught behind for 31 with a spiteful delivery that lifted sharply off a length.
Bavuma, returning to the side from a calf injury after missing the tour to Pakistan, lasted 11 balls before spinner Kuldeep Yadav had the visiting skipper caught at leg-slip for three.
Tony de Zorzi and Wiaan Mulder then took the Proteas to lunch with no further damage, and would have been reasonably pleased with the total on 105-3.
De Zorzi and Mulder extended their partnership to 43 before the latter chose the wrong ball to reverse-sweep and was trapped lbw for 24.
The left-handed De Zorzi (24) then suffered a similar fate to Bumrah with the tourists going on to lose five wickets for 49 runs in the second session.
Bumrah completed his five-wicket haul after the tea interval and finished with figures of 5-27 in 14 overs while Kuldeep took 2-36.
Marco Jansen gave the Proteas an early breakthrough after bowling opener Yashasvi Jaiswal off the inside-edge in the seventh over but Rahul and Sundar went unscathed until bad light brought an early end to proceedings.