Make custom versions of OpenLayers which include only the parts that you want to use. Currently we're baking up OpenLayers 2.9.1, the latest stable release.
For when you just want OpenStreetMap.