A man is suing Wisconsin Children’s Hospital after they wouldn't let him see his sick son.

Time reports that Stuart Yates, who's a registered sex offender, claims the hospital treated him unfairly when they denied him visitation with his son. "They’re using my past record, 20 years ago, against me,” he said. "I served my time for it. I’ve been transparent about it."

Yates' lawsuit claims his son's blood infection caused by e-coli was just the latest of serious medical issues his 9-year-old son has faced. But this is the first time that a hospital denied him visitation. It was especially worse since Yates' son was scheduled for surgery.

"He called me this morning and he was crying on the phone," Yates said, "telling me how much he needed me and how scared he was and where am I at? 'Dad where are you? How come you won’t come? I miss you, I need you.'"

Hospital spokesman Andy Brodzeller has since shared that rules can be enacted if it's for the benefit of their patients. "The hospital does have visitation policies in order to ensure the safety of our patients, visitors, and staff and those practices do allow us to take steps that would limit an individual’s access to our hospital," he said.

But Yates' lawyer, Adele Nicholas, says that it's up to the patients to decide who they can see. "The law is quite clear that the patient has a right to designate the people they want to visit with while they’re in the hospital and the hospital is not supposed to interfere with that visitation," she said.

Yates was convicted of second-degree sexual assault of a child in 1998, according to Time. Since then Yates has maintained that he did not know the age of the victim and that it happened "at a house party where there were strippers." He later made a deal with prosecutors and was sentenced to six months' time served.