Were the Muhadithun Just Following Their Desires When They Collected and Graded Ahaadeeth?
The answer is no, and the burden of proof is on the one alleging this. There are good reasons for believing that the Muhadithun were sincere, and we list a few reasons briefly here:
1) Their endurance of hardships and trials while collecting the ahaadeeth
2) Implausibility of "coincidental desires"
3) Their lack of personal bias when grading the ahaadeeth
4) Their willingness to admit their mistakes
5) Their establishment and abidance of principles of Jarh wal-Ta'deel and ahaadeeth authenticity verification.
Let us briefly discuss each point.
1) Their endurance of hardships and trials while collecting the ahaadeeth
There were no airplanes and cars back in those days. The Muhadithun had to undertake great hardships when traveling on very long journeys back then. They would have to spend from their own wealth, sacrifice being away from their families for long periods, and endure the physical hardships of traveling long and rugged terrains. They would not always be riding their camels; they would do a fair share of walking on their feet as well.
Sometimes they do this only for the sake of verifying a single hadeeth! If they were following their desires, why go through all of that?
2) Implausibility of "coincidental desires"
As we know, several ahaadeeth fortunately benefit from being multiply attested. If the Muhadithun were just following their desires, how likely is it that several Muhadithun would independently have the same “desires” in concocting the same hadeeth? This is not very likely.
3) Their lack of personal bias when grading the ahaadeeth
We know of several cases whereby the Muhadithun would accept ahaadeeth from Muslims they deemed innovators. If a Muhadith were convinced that the innovator (whether he be from the Shia, Khawaarij, etc.) was telling the truth when narrating the hadeeth, he would accept from him. The Muhadithun made sure that sectarianism did not come in the way of preventing the truth from coming out.
On the flip side, we also know of cases whereby the Muhadithun would criticize people deemed very close to them. Imam Abu Dawud famously criticized his own son Abdullah as a liar! Does this sound like someone following his desires?
4) Their willingness to admit their mistakes
The Muhadithun were known for admitting their mistakes and making retractions, whether they be found in the same book (in a later chapter) or in a different book written later.
That does not sound like an egoistic person who just follows his own desires, now does it?
5) Their establishment and abidance of principles of Jarh wal-Ta'deel & ahaadeeth authenticity verification
The Muhadithun set up stringent guidelines to follow when determining the status of every single narrator of hadeeth to the best of their abilities. They would most importantly first find out if the narrator is trustworthy and of upright status before proceeding on to other critical pieces of information (e.g., the accuracy of memory, where he lived and whom he heard from during what stages in his life, etc.). They would then classify the narrators into different categories accordingly.
In addition, a critical examination of the sanad (i.e., chain of transmission) would take place. They would ensure that the sanad is fully connected and find out whether the narrator heard from the other narrator via hearing directly or indirectly, reading his books, etc. They would ensure whether the hadeeth goes back to the Prophet, Sahabi, etc. and, as a result, give the hadeeth its stature of religious authority. Overall, they would ensure the sanad is free from all possible defects, and one could learn about all the possible defects a sanad may be liable to when studying the hadeeth sciences.
Furthermore, the Muhadithun would also critically examine the matn (i.e., the actual content and wording of the hadeeth). They would ensure whether the matn is Saheeh in its meaning, whether it conflicts with other soundly transmitted ahaadeeth and Qur’anic verses, whether it is narrated verbatim or paraphrased, whether the hadeeth is an abrogator or abrogated, etc.
Given all this, we ask: why on earth would the Muhadithun go through all this hassle just to end up following their desires at the end of the day?
Conclusion
Of course, being "sincere" does not make the person right; however, I hope the above points (despite being greatly summarized and is my hope would stimulate people to research further) at least eliminated one potential doubt, and that is the question of the sincerity of the Muhadithun.
Allah chose these Muhadithun to serve a great cause for Islam, and we should be thankful to Allah for them. May Allah reward them greatly for their efforts.