Shalin Ren Payne, 29, was reportedly babysitting his fiancee's two children while she was at work on Oct. 5 when he became violent and attacked the 12-year-old girl.
According to The Atlanta Journal-Constitution, Payne, who has tattoos covering his entire face, was released on parole after serving two years in prison for a burglary conviction.
Prior to that charge, he had served four and a half years behind bars between 2004 and 2012 for charges including burglary, aggravated assault and theft.
Coweta County sheriff's deputies arrested the man in Sharpsburg, Georgia, after responding to a 911 call around 8:30 p.m., according to the Times-Herald.
Payne reportedly chased the 12-year-old girl through the house, breaking down the bathroom door after she locked it in an attempt to get away from the felon.
Payne threw the little girl on the floor, held her down, and choked her, while her 4-year-old brother sat just outside of the bathroom, unable to do anything to stop the attack, which lasted less than an hour, Lt. Col. James Yarbrough from the Coweta County Sheriff's Office told Times-Herald.
When the mother of the children came back from work, she said she found Payne on top of her daughter, and slammed a glass dish on his head in an attempt to stop the assault. Payne then reportedly assaulted the mother before fleeing the home wearing nothing but a T-shirt.
After receiving the 911 call, authorities apprehended Payne, who had attempted to escape the deputies on foot, Yarbrough told Times-Herald.
Payne has been arrested on charges of aggravated sodomy, aggravated assault, sexual battery and rape, and was taken to the Coweta County Jail, where he may face additional charges.
The 12-year-old girl was taken to two hospitals to receive treatment and for further examination.