So I came across another reason for the Flex #2032 error. In my case, it wasn't the crossdomain.xml that was the problem. This was an IE7/8 specific issue, Firefox was working fine. After a lot of head banging I found out this about Flex:
Flex strips the request of headers if you make a GET request. So if for some reason the server requires headers (like in my case), you will get a #2032 error. Just change it to a POST request if you have headers to send.