Both empires had their good times and bad times