I know they look exactly the same, but did the Chimer just abandon magic and decide to go for a more warrior type culture? I know they still retained some form of magic but im asking what the difference culture wise was.

Perhaps over time the Chimer/Dunmer became a more warrior type culture than the Altmer, but that's not what originally differentiated the two. Simply put the Chimer were Altmer who followed the teachings of Veloth and began to worship Daedra instead of Aedra.

Culture wise the chimer strongly worshipped the daedra and that led to their war with the dwemer Because the the dwemer thought through Intelligence they could become stronger than gods something along those lines

Where does the assumption come from that the Dunmer as a whole became a "far more warrior type culture"?

Have a look at the differences between the Great Houses. The Telvanni are just as magically inclined as their Altmeri ancestors.

The Hlaalu are merchants and not widely known as a "warrior culture".

House Redoran certainly fits your bet, but the ancient Aldmer also had their warrior sects, for example the Mer who followed Trinimac and later became the Orsimer.