Organic matter in the environment represents a complex mixture of materials that are broken down on different timescales. Fresh material released by living or recently deceased organisms is generally degraded within days to weeks, while a large portion resists being broken down and can last for thousands of years. Scientists have long been challenged to explain what make some organic matter so persistent. Using newly developed analytical tools, we measured the abundance of thousands of molecular components of this organic matter pool with depth in the North Atlantic Ocean. We then sorted these molecules into different groups based on their lifetimes in the ocean, with more persistent chemical forms accumulating to a greater extent in deeper waters far from where they are produced in the sunlit surface. We discovered key chemical differences between the short-lived and long-lasting molecules. The short-lived organic matter, found near the ocean surface, was less water soluble than the persistent material found uniformly at all depths. This finding suggests that the 'greasy' nature of these molecules causes them to be removed more quickly by aggregating or sticking to particles. This process is a crucial factor in controlling how long carbon is stored in the ocean.