Lord Dearing, a former Post Office chairman who has produced a series of major studies on education for successive governments, said that ministers may have to reverse their decisions to make languages optional for the over-14s if the "severe" decline continues.
His report said the decision had "undermined" efforts to improve language teaching.
Lord Dearing said that the widespread perception that language GCSEs are more difficult than other subjects may have contributed to its decline because schools shun the subjects to bolster their standing in league tables.
His report said: "There has been long sustained argument that the standards for the awards of grades are more demanding than for other subjects, and that this has contributed to the flight from languages, both because of the concern of students to get good grades and the concern of schools to do well in the 5 A*-C achievement and attainment tables. This is a continuing sore point."
Since compulsory languages were axed in 2004, the proportion of pupils taking a foreign language at GCSE level has fallen from 80 per cent to just 50 per cent.
Lord Dearing said that a return to compulsory language teaching for 14 to 16-year-olds was not his "preferred" option, but said it "should be used if it proves to be needed".
Instead he called for languages to become a "standard" part of the national curriculum in primary schools to "respond to the enthusiasm and ability to learn through games and play at primary level."
He said: "For languages, the earlier the better. We like the way they are being taught in primaries as they are introduced through cross-curricular work, and the way they draw on the young children's sense of fun. We propose that they should be embedded in the primary curriculum."
Lord Dearing also called for regulations forcing schools to teach at least one European language to be lifted, to give pupils a much wider choice of non-European languages.
He said: "For today's young people, languages matter: they are an investment that can enrich their lives socially, culturally, and economically. There is a significant danger that if some pupils - particularly low achievers - are restricted to a monolingual, monocultural education they will be increasingly unable to deal with the complex demands of our society."
Alan Johnson, the Secretary of State for Education, welcomed the study.
But David Willetts, the shadow Education Secretary, said languages must be taught "to all pupils in every school". He said: "Opting out from modern languages in schools means pupils are opting out from opportunities in life."
