Researchers looked at 500 Israelis who had developed the condition and compared their mobile phone usage with 1,300 healthy controls.
Those who had used the phone against one side of the head for several hours a day were 50% more likely to have developed a salivary gland tumour.
The research appeared in The American Journal of Epidemiology.
Numerous studies have focused on the risk of tumours among those who use mobile phones, and overwhelmingly found no increased cancer risk.
But researchers at Tel Aviv University say these have tended to focus on brain tumours, and often did not include long-term users.
Cancer of the salivary gland is a very rare condition. Of the 230,000 cases of cancer diagnosed in the UK for instance annually, only 550 relate to this area.
Dr Siegal Sadetzki, who led the research, said while mobile phone use in Israel was much heavier than in many other parts of the world, this gave an insight into what the long-term, cumulative impact could be.
"Compared to other studies, the amount of exposure to radiofrequency radiation we saw here was much higher. If you like, you're seeing what could happen elsewhere 'speeded-up' in Israel," she said.
One of the key findings of the study was that heavy users in rural areas had an even higher risk that those in cities, due, the team suggested, to the fact that mobile phones in areas without strong signals need to emit more radiation to work properly.
But Dr Sadetzki stressed one study was not enough to prove a link, and that further research was needed.
Nonetheless, until more evidence became available, a "precautionary" approach was best, she said, particularly when it comes to children's use of mobile phones.
Despite these latest findings, the largest and longest-running investigation ever to be carried out into mobile phone usage found no increased risk of any sort of cancer.
It followed 420,000 people in Denmark, some of whom had been using a mobile phone for as long as ten years.
There was in fact a lower incidence of cancer than expected in a group of that size, suggesting mobile phones had no impact on the development of tumours.
Last year, the UK's Mobile Telecommunications and Health Research Programme said that while the evidence so far was "reassuring", there was still a need for studies to examine the very long-term impact, and to look at the effect in children.
Ed Yong, of Cancer Research UK, said: "Mobile phones are a relatively recent invention and new research into any possible health risks is welcome.
"However, it's important to remember that the vast majority of studies so far have found that mobile phones do not increase the risk of any type of cancer."