This is indeed one of the conclusions that Yitzhak Ben-Zvi—a historian and ethnographer who would later become Israel’s first President—reached from studies of the names and traditions of many Palestinian villages and Bedouin communities.
He wrote about this extensively in a book that he wrote around a century ago, jointly with David Ben-Gurion (who would later become Israel’s first Prime Minister)—Eretz-Israel Ba’Avar Ubahoveh (“The Land of Israel in the Past and in the Present”), and in another book of his own.
Essentially, the indications are that Jews who had remained in Palestine or who had returned to it after the brutal Roman suppression of the Jewish revolt in the early 2nd century CE, remained Jewish until 1012 CE, when the Caliph el-Hakim forced all non-Muslims to convert to Islam. (Since Islam is a true monotheistic faith, conversion to it was not considered under Judaism a heretical transgression of the type ייהרג ובל יעבור — i.e. one that they must “die to avoid doing”—so most, if not all, chose to do so rather than be killed.) Some Spanish Jews who arrived in Palestine a few centuries later were similarly obliged to convert by the Mamluk rulers of that time.
Although few if any of their descendants chose to convert back to Judaism in the centuries that followed, many continued to observe various Jewish customs, such as lighting candles on Friday nights, and only ceased to do so in the 1930s and ‘40s, when neighbouring Arabs scolded them for such Jewish practices because of the growing nationalist tensions with the Jewish community.
Estimates vary as to the percentage of Palestinian Arabs who are of Jewish descent, and undoubtedly many are not, but the evidence is there. Which puts a whole new complexion on the modern Hebrew euphemism for an Arab —ben-dod (“cousin”)…
For more information, read, for example:
(Originally written in reply to a question at Quora.com)
Notes by Autumn Light 