Police spokeswoman Luba Samri said anti-Christian slogans were found on the outer walls of the Dormition Abbey, a Benedictine monastery located just outside Jerusalem's Old City and where Christian tradition says the Virgin Mary died.
The graffiti included threats of violence, messages degrading Jesus and a call for Christians to "go to hell".
Nikodemus Schnabel, the Dormition Abbey spokesman, said the graffiti bore "very radical messages".
Police are investigating the incident but suspicion immediately fell on Jewish extremists who have for years vandalised Palestinian property, as well as mosques, churches, the offices of dovish Israeli groups and even Israeli military bases.
The so-called "price tag" attacks seek to exact a cost for Israeli steps seen as favouring the Palestinians.
That attacks have prompted widespread condemnation and pledges by Israel's government to get tougher on Jewish vigilantes.
Israel's Minister of Public Security, Gilad Erdan, said he has instructed police to give the vandalism case top priority.
"We will not allow anyone to undermine the coexistence between religions in Israel. We will show zero tolerance to whomever harms the democratic foundations of Israel and its freedom of religion and we will apprehend those who carried out this heinous act," he said.
Prime Minister Benjamin Netanyahu also condemned the vandalism.
The Benedictine abbey is a popular site for pilgrims and tourists. It has been damaged several times in recent years.
The vandals are believed to be from the same group of Jewish extremists who have carried out a spate of similar attacks in recent years in retaliation for Palestinian attacks or to protest what they perceive as pro-Palestinian policies by the government.