Huntington's disease is one of several neurodegenerative disorders characterized by the aggregation of polyglutamine (polyQ)-expanded mutant protein. How polyQ aggregation leads to cellular dysfunction is not well understood. Here, we analyzed aberrant protein interactions of soluble oligomers and insoluble inclusions of mutant huntingtin using in-cell single molecule fluorescence spectroscopy and quantitative proteomics. We find that the interactome of soluble oligomers is highly complex, with an enrichment of RNA-binding proteins as well as proteins functioning in ribosome biogenesis, translation, transcription, and vesicle transport. The oligomers frequently target proteins containing extended low-complexity sequences, potentially interfering with key cellular pathways. In contrast, the insoluble inclusions are less interactive and associate strongly with protein quality control components, such as Hsp40 chaperones and factors of the ubiquitin-proteasome system. Our results suggest a "multiple hit" model for the pathogenic effects of mutant huntingtin, with soluble forms engaging more extensively in detrimental interactions than insoluble aggregates.