The 19-year-old "Isis soldier" was being monitored with an electronic tag after he was arrested for twice attempting to flee France to join the terror group in Syria.
Despite having been released early from prison, Keriche's bail conditions allowed him to roam freely between 8.30am and 12.30pm. The attack happened between 9am and 11am.
Both were shot dead by police marksmen as they emerged from the building shouting 'Allahu Akbar' following the attack that also left a nun critically injured.
A third man, a 17-year-old known as HB and believed to be a relative of Kermiche, was arrested at his home in Saint-Etienne-du-Rouvray following the attack.
The revelation - made to the French TV news channel I-Tele - will cause further outrage in a country devastated by constant security failings.
French President Francois Hollande, who visited the scene today, said the country is now 'at war' with ISIS after the terror group claimed responsibility for the attack.
Islamic State has claimed responsibility for the savage attack. According to security sources, one of the attackers was a 19-year-old convicted terrorist living with his parents and who forced to wear an electronic tag, The Sun reports.
The Catholic church was also on a terrorist "hit list" that French police have reportedly known about since April 2015 following the arrest of an extremist in Paris.
Tuesday's chilling assault was the first known attack inside a French church in recent times. A church was targeted last year, but the attack was never carried out.
The hostage stand-off occurred at the Saint-Etienne parish church, when the two beared men with knives entered the church through a back door and took five people hostage - including a Catholic priest, two nuns and two other worshippers - for at least an hour.
Gunfire was heard, and the men were shot and "neutralised" by police called to the scene.
Interior Ministry spokesman Pierre-Henry Brandet said police managed to rescue the only three other people inside the church.
According to French media reports in Le Point, sources said the assailants shouted "Daesh", an Arabic word in its own right meaning 'a group of bigots who impose their will on others'.
A sixth person reportedly escaped and managed to raise the alarm of a hostage situation underway.
The RAID special intervention force was searching for possible explosives in or around the church.
"The investigations are ongoing. There are still unknowns," Brandet said. "There are dogs, explosive detectors and bomb disposal services and as long as there are still unknowns, the judicial police cannot get inside the site. It's a dramatic situation."
"I cry out to God, with all men of good will. And I invite all nonbelievers to unite with this cry," Lebrun wrote in a statement from Krakow, Poland, where Pope Francis was expected.
"The Catholic Church has no other arms besides prayer and fraternity between men."
French President Francois Hollande, arriving on the scene, called it a "vile terrorist attack" and said it's one more sign that France is at war with IS, which has claimed a string of attacks on France.
"We must lead this war with all our means," he said, adding that he was calling a meeting on Wednesday of representatives of all religions.
He expressed solidarity with local Catholics, saying "they have been terribly hit by the killing of the parish priest by two terrorists claiming to belong to Daesh. I have met with the family of the priest."
A police official said one of the attackers was turned back after trying to go to Syria. The official, who spoke on condition of anonymity to reveal details of the investigation, said the man wore an electronic bracelet to monitor his movements.
Mohammed Karabila, head of the Regional Council of the Muslim Faith for Haute-Normandie, said French security services knew the name of one of the attackers.
"The person who committed this odious act is known and he has been followed by the police for at least 1.5 years. He went to Turkey and security services were alerted after this," he told The Associated Press by phone. He refused to divulge man's name and had no information on the second attacker.
Pope Francis condemned the attack in the strongest terms.
Vatican spokesman, the Reverend Federico Lombardi, said in a statement the attack hits particularly hard "because this horrific violence took place in a church, a sacred place in which the love of God is announced, and the barbaric murder of a priest."
France is on high alert and under a state of emergency after an attack in the southern city of Nice on Bastille Day - July 14 - that killed 84 people that was claimed by Islamic State, as well as a series of attacks last year that killed 147 others around Paris.
French Prime Minister Manuel Valls tweeted: "I am horrified by the barbaric attack on the church in Seine-Maritime. All of France and her Catholic citizens have been wounded. We stand in solidarity."
Interior Ministry spokesman Pierre-Henry Brandet said on France-Info radio that anti-terrorism investigators have been summoned in the case.
Police checked the church for explosives or booby-traps and forensic officers are taking fingerprints and DNA evidence, he also said.
Eulalie Garcia, who works in a beauty parlour on the same road as the church, told reporters that she knew the priest, who had taught her the catechism as a young girl.
"My family has lived here for 35 years and we have always known him," she said.
"He was someone who was treasured by the community. He was very discreet and didn't like to draw attention to himself."
She said she was very shocked by the death of the priest, who lived opposite his church.
Islamic State extremists have urged followers to attack French churches and the group is believed to have planned at least one church attack earlier.
In April 2015, an Algerian student who was arrested after shooting himself in the leg was found with heavy weapons, bulletproof vests and documents linked to Islamic State.
He is charged with killing a young woman inside her car the same day. According to French authorities, the suspect, Sid Ahmed Ghlam, was sent by the Belgian Abdelhamid Abaaoud to attack a church in Villejuif, just outside of Paris.
A cell directed by Abaaoud later carried out the November 13 attacks in Paris that left 130 people dead and the March 22 attacks in Brussels that killed 32 people.
