In international achievement studies, a common test is typically used which is translated into the languages of the participating countries. For the test to be valid, all the translations and different-language test versions need to be equally difficult to read and answer. An underestimated and underdiscussed threat to this validity is unwanted literal translation. This paper discusses the problem of unwanted literal translation in international achievement studies. It defines what is meant by unwanted literal translation and explains why it is a threat to the validity of international achievement studies and why it is so difficult to avoid. It also discusses problems there have been when translating these tests which may have promoted unwanted literal translation and provides suggestions on how to improve the translation practices so as to ensure that the translations are in as natural and idiomatic language as possible.