I thought it would be a good Idea to include a post that provides information on the troubles caused by removing a RD Connection Broker Availability configuration.
Once a RD Connection Broker HA configuration is installed, you cannot revert back to the windows internal Database with out decommissioning the whole RDS configuration.
When you add the default connection broker Role, the Windows Internal Database will be used.
You can connect to the Internal Windows Database using SQL Management Studio Express & Named Pipe
When you configure a RD Connection Broker HA, the internal database data is migrated over to the SQL instance you have specifyed. Please see the link below that shows the data inside a RD connection broker database. http://social.technet.microsoft.com/wiki/contents/articles/16246.a-closer-look-at-the-rd-connection-broker-database-in-windows-server-2012.aspx
When you try to remove the connection brokers, you would need to be aware that all the data and RDS configuration would be Lost. This means you would loose the configuration from all roles and you would see the following:
Once a HA is configured, you are stuck with it unless you want to rebuild everything.