After struggling with this for almost a day, I'm posting here for future reference:
When you search for a crawled property, you may find two crawled properties that represent the same content. For example, a site column of type Text named Color will during crawl discover two crawled properties: ows_Color and ows_q_TEXT_Color. Crawled properties that begin with either ows_r
, ows_q , or ows_taxId are automatically created crawled properties.
When you select a crawled property to map to a refinable managed property, make sure that you don't map the automatically created crawled property. Instead, always map the crawled property that begins with ows_.
https://technet.microsoft.com/en-us/library/jj679902.aspx#BKMK_MapCPtoRefinableMP