Talk to General Physician on Fish With Milk

Can we drink milk after fish. Is there any harmful effects of it on body or not. Almost in all cultures it is said that it can be harmful eating fish and milk or curd. What medical science says about it? Thanks Waris