The Compute and control for adaptive optics (Cacao) is an open source software package providing a flexible framework for deploying real-time adaptive optics control. Cacao leverages CPU and GPU computational resources to meet the demands of modern AO systems with thousands of degrees of freedom running at kHz speed or faster. Cacao adopts a modular approach, where individual processes operate over a standardized data stream stucture. Advanced control loops integrating multiple sensors and DMs are built by assembling multiple such processes. High-level constructs are provided for sensor fusion, where multiple sensors can drive a single physical DM. The common data stream format is at the heart of Cacao, holding data content in shared memory and timing information as semaphores. Cacao is currently in operation on the general-purpose Subaru AO188 system, the SCExAO and MagAOX extreme-AO instruments. Its data stream format has been adopted at Keck, within the COMPASS AO simulation tool, and in the COSMIC modular RTC platform. We describe Cacao's software architecture and toolset, and provide simple examples for users to build a real-time control loop. Advanced features are discussed, including on-sky results and experience with predictive control and sensor fusion. Future development plans will include leveraging machine learning algorithms for real-time PSF calibration and more optimal AO control, for which early on-sky demonstration will be presented.