Negative evaluations of figures like Thomas Jefferson are prone to the charge of presentism, i.e. forming views according to present sentiments and not according to those of the society in which the object of study lived. According to this train of thought, we should analyze the role that Jefferson played strictly as understood by the society in which he lived, not part of a larger study of that society.  This criticism is deficient. In all historical studies certain appraisals are made, though often implicitly, of the society in which the object of study lived. The field of history is not simply the sequencing of events and assigning causal significance. Rather, historians routinely and inextricably assign value-based significance to these past events, typically based on (or at least mediated by) prevalent views and mores of the society in which they are writing. Given this, historical narratives which praise Jefferson are more accurately committing this sort of presentism as they often hinge on a current (though not eternally) favorable view of the United States during his period. 
Under scrutiny are Jefferson's writing and policies as they directly related to colonized people and foreign powers. These serve to illustrate not that Jefferson himself was some horrible 'racist' (which would denote an overriding personal quality), but that he led a society which was fundamentally characterized and driven by its overt, structural oppression of others.
Class structure of the nascent United States
The United States, unlike most other countries, was founded as a country of property owners. This property mostly came in the form of other people, i.e. African slaves, or land recently taken from Native Americans. The economy at large was structured around these two factors, and this form of social-relations (settler-colonialism, for lack of a better term) produced its own distinct ideological manifestations.
Racially-justified slavery was not some incidental fact in U.S. history. Prior to the successful rebellion against England, approximately one in five people in the colonies was an enslaved African. This social and productive relationship was a cornerstone of the newly-birthed U.S. For his part, Jefferson expounded both conceptual frameworks of justification and offered stern warnings over the inherent dangers of such a system.