The huge red crane crashed into a part of the Grand Mosque - the largest in the world - that was filled with worshippers at the time.
The head of Saudi Arabia's civil defence said strong winds and heavy rains had caused the collapse.
Mecca is currently preparing for the annual Muslim Hajj pilgrimage.
Hundreds of thousands of people are expected to arrive in the Saudi city from all over the world later this month.
The collapse happened at 17:23 local time (14:23 GMT), said Director-General of the Saudi Civil Defence Authority, Lt Sulayman Bin-Abdullah al-Amr.
It happened at a time when many worshippers were there for Friday prayers.
The toll for the dead and injured, tweeted by the Civil Defence Authority (in Arabic), continued to rise throughout the evening.
Shortly before the crash, the city had been hit by unusually high levels of rainfall and winds of up to 83kph (50mph), Lt Amr said.
A video posted on YouTube, which could not be independently verified, appeared to record the moment the crane fell, with a loud crash heard in the background followed by panic and shouting.
Images circulating on Twitter showed what looked like numerous bodies and blood on the floor of the mosque.
Lt Amr said an investigation was being carried out to assess the damage, and the "extent of the safety of these sites".
Mecca is preparing to welcome Muslims from around the world for the annual Hajj pilgrimage, which begins in about 10 days' time.
Islam requires that every Muslim capable of doing so performs a pilgrimage to the site at least once in their lifetime.
Saudi authorities began a major expansion of the site last year to increase the area of the mosque by 400,000 square metres (4.3 million square feet), to allow it to accommodate up to 2.2 million people at once.
More than three million people undertook the Hajj pilgrimage to Mecca in 2012. Saudi authorities have taken steps since 2013 to limit the number of people involved.
Large numbers of people have resulted in several tragedies over the years, including a stampede in 2006 that killed nearly 350 people.