Reflections on Nihongo Overlapping: A Tool for Evaluating Japanese Speech Learning.

Dr. Hiroaki HATANO
Assistant Professor of Humanities and Social Sciences, University of Tsukuba
Born in Kyoto, Japan. Affiliate Researcher, Hiroshi Ishiguro Laboratories at the Advanced Telecommunications Research Institute International (ATR). His research fields are Japanese linguistics and Japanese language education. His specialties include speech science and speech communication.
Nihongo Overlapping is an automatic evaluation tool designed to support Japanese learners in their speech practice. No complicated operations are required; you can easily practice and evaluate Japanese prosody1) by simply "hearing, watching, and imitating" the model audio. We interviewed Dr. Hatano, a specialist in speech science who developed this tool, to learn about the development background and how to use it.
1) Prosody refers to aspects of speech sucha as an accent, intonation, and rhythm. In Japanese speech, the correctness and naturalness of prosody are considered more important than individual phonemes.
Overcoming Challenges for Japanese Language Teachers with the Support of Technology
First of all, please tell us why you decided to create Nihongo Overlapping.
HatanoWell, I specialize in speech science, but I have been thinking for some time about the possibility of creating a tool to evaluate the speech of Japanese language learners. In Japanese language education, there are relatively few opportunities to address speech, and I think a reason for this is that, unlike writing, speech is not visible to the eye, and evaluation itself is difficult. It is said that Japanese people also have difficulty pronouncing L and R, and we wondered if we could overcome this difficulty with the support of technology.
Why is it difficult to evaluate speech?
HatanoEvaluating speech can be difficult because teachers of Japanese, particularly those teaching non-native speakers abroad, may struggle to accurately assess Japanese pronunciation. Furthermore, the presence of numerous regional dialects among native Japanese speakers adds to the challenge.Therefore, I believed that having a tool capable of properly visualizing, comparing, and evaluating speech could help overcome these difficulties.
How long did it take to develop this tool from start to finish?
HatanoThe concept itself has been in the works for some time, but actual development did not begin until FY2022. It was not until the middle of FY2022 that the tool took shape. The completion was in July 2023, so it took about a year from the start to completion. I was the main person in charge of the project, but the actual engineering implementation was supported by the developer, Carriage, Inc..
No matter the level of the learner, they can practice with any audio source they like
Please tell us the target user (level and purpose) for this tool.
HatanoSince this tool allows you to practice with your favorite audio source, the level of the learner is actually not that important. Whether you are a beginner or an advanced learner of Japanese, you can practice according to your level. Beginners can use the soundtracks from textbooks and other audio to practice oral exercises, while advanced learners can practice their speech and other skills.
In particular, I think this is ideal for learners studying abroad who may not have the opportunity to have someone else listen to their speech.
Please tell us the features of this tool.
HatanoOne feature is that everything can be done on the web.There has been specialized software to generate pitch curves (pitch of sound) for speech, but it usually has to be downloaded before using. I think the fact that this is not necessary is a big advantage.
Another feature is that the prosody can be visualized quickly with ease. With a single glance, you can see the intensity, height, and length of the sound.
One more significant feature is that it gives you a score. You can superimpose the model speech and your own speech and get a score for the phase of the matched parts. This provides extra motivation, I think.
However, it is impossible even for Japanese to achieve a perfect match, so you don't need to aim for perfection. I think it is best used as a tool to get a sense of how far off you are.
What kind of usage do you envisioned for this tool?
HatanoBasically, we expect the learner to proceed with learning on their own. I think there are quite a few people who feel embarrassed to have their speech evaluated. Even Japanese people are embarrassed to have their English pronunciation heard, so we think it would be good if they advance by themselves.
However, it is not as if it cannot be used in the classroom. For example, the teacher could prepare an audio source in class and use it as homework for learners, asking students to practice a little oral familiarization with it, and so on. Teachers can request students to practice until they reach a certain level like 80 points.
There is not really a strict usage plan, and I hope that people will enjoy using it.
It seems useful for entrance exams, job interview practice, and various other purposes.
HatanoYes, that's right. Of course it is important to convey information accurately using your voice, but the way you are perceived is also important, so I think it is fine if you use this tool for situations such as job hunting.
Specializing in very straightforward areas by starting with prosody
What were the key considerations in developing this tool?
HatanoFirst of all, we did not use technical terms. We were careful not to use jargon, but to make it intuitively understandable at first glance. Some people may have a negative impression of phonetic sounds because they have an image of something unusual and difficult, such as memorizing IPA2) or looking at a cross-sectional view of the inside of the mouth. Therefore, we tried to make it as easy to understand as possible in terms of appearance.
I am not at all saying that this tool provides the pronunciation of each and every one of these sounds. The tool starts from the prosody, that is, it gives priority and focuses on the most straightforward part of the sound, the pitch of the sound. I feel that if the students start from this point, they will have less difficulty in understanding the sound.
2) IPA: International Phonetic Association.
More Convenient When Linked to OJAD
Please give some advice to Japanese language teachers who plan to use this tool in their teaching.
HatanoBasically, I think it would be best if the teacher introduces it to the learners and then they can use it as they wish. For example, I think it would be better if it is used in conjunction with OJAD3) , which is already ahead of the curve and everyone knows about.
OJAD has a very rich collection of audio resources. The OJAD has the ability to create a model voice by inputting text4) , so I think it is possible to use it in combination with OJAD in any number of ways that are appropriate to the site or environment where teachers are currently teaching, depending on their creativity.
3) OJAD (Online Japanese Accent Dictionary) (https://www.gavo.t.u-tokyo.ac.jp/ojad/)
4) One of OJAD's four functions, Prosody Tutor Suzuki-kun, creates a model speech by entering text. (https://www.gavo.t.u-tokyo.ac.jp/ojad/phrasing)
I recommend "Japanese Language Magnet"
Do you have any recommendations for Japanese language educational content on JV-Campus?
HatanoI really like "Japanese Language Magnet". The tempo is good and the length of around 10 minutes is also good. The videos themselves are stylish and the animation is enjoyable. The audio is also easy to follow, and I think the way it is made, like NHK's educational programming, is quite good.
Speech communication is inevitable in studying abroad and finding a job
Finally, do you have a message for Japanese language learners who are aiming to study or work in Japan?
HatanoWe understand that there are many different backgrounds of people who want to study abroad or find a job, and spoken communication is an inevitable part of the process. We hope that you will use these tools to enjoy learning and not be intimidated by the idea of speaking.
Thank you very much for the interview Dr. Hatano.
Related content
How to use the tool
I. How to set the model speech
How to set the model speech
- 1Give your task a title.
-
2Choose the type of speech material you want to use a model speech.
Prepare a model speech file that you would like to practice with from the CD that came with your textbook (Files with the extension wav, wevm, mp3, or mp4 are required). - 3Enter the text of your model speech.
- 4Record your voice and get evaluated!
II. Recording and Evaluation
Recording and Evaluation
- 1First, play and listen to the model speech. Pay attention to changes in its prosody.
- 2When you are ready, start recording. Aim for the prosody of the model speech.
- 3You can also your score.
- 4Your recording result is displayed in green, superimposed on the model speech. Check how it differs from the model.
Let's aim for a score of 100 point!
You can practice as many times as you want with the "Retry recording" option.