The film begins with children singing a song about a man who chops up little girls. A woman carrying laundry walks by the children and yells at them to stop singing that horrible song. The children stop for a few seconds then start singing again. The woman with the laundry stops by Mrs. Beckman's apartment. Mrs. Beckman is waiting for her daughter, Elsie, to get home from school, and starts setting the table for lunch.
The scene cuts to where Elsie is standing outside of the school. She is playing with a ball, bouncing it off of a poster on a telephone pole. The poster says "Who is the Murderer?" Then the shadow of an adult falls over the poster. The adult asks Elsie who she is and compliments her on her ball. We cut to Mrs. Beckmen looking at the clock worriedly. She asks some children outside her door if they have seen Elsie. They have not. The camera cuts again to Elsie and the man by the poster buys her a balloon. While he is buying the balloon from the blind salesman, he whistles In the Hall of the Mountain King by Grieg. At home, Mrs. Beckman feels that there is something wrong and starts yelling for her daughter out the window. Her calls turn into screams as the camera shows the empty stairwell, empty basements, then shots of Elsie's ball rolling in the grass and her balloon caught in the telephone lines. Lastly the scene cuts to a paper boy calling out on the street corner the news of the child murderer.
The next scene shows someone writing in pencil a letter to the newspapers saying he has not reached the end. There will be more to come.
The scene cuts to the police station. The government minister calls the chief of police who reports the police's progress in thier pursuit of the child murderer. They have been making raids on the local hangouts of the underworld. Commissioner Lohman sets out into the rain with his comrades to arrest criminals at The Crocodile .
The leaders of the underworld do not like how the police keep interrupting their activities looking for the murderer. So, Schr�nker, head of underworld, sits down with the other leaders to try to figure out what to do about the situation. At the same time as their meeting, the police are also having a meeting to discuss the child murderer. The camera switches back and forth between the police and the underworld as the man in one group finishes the sentence of the man in the other group. This effect demonstrates the parallels between the thoughts of the two groups. Finally, the underworld leaders decide to send the beggar's organization out to watch the children and look for the murderer that way. Lohman has the idea of looking up mental institution files for a person who has been let out and labeled "harmless".
The police figure out the name of the murderer, Beckert, by looking through the files and searching his house. They find pencil shavings on the window sill that are the same color as the pencil that was used to write the letter to the newspaper. The police wait in Beckert's house for his return home to catch him.
Meanwhile, Beckert is eating fruit on the street while looking at knives through the display window of a knife shop. As Beckert is looking at the knives he sees the reflection of a little girl, who is standing right next to him, and starts to tense up. Beckert follows the little girl as she walks away, and starts whistling In the Hall of the Mountain King . The child's mother finds the child just before Beckert is able to get to her. Still excited by the sight of the little girl, Beckert goes to an outside cafe to have a drink.
On the street, Beckert is identified by the blind beggar that had sold Beckert the balloon on the day Elsie died. The beggar recognizes the tune that Beckert is whistling while Beckert leads another little girl to a candy shop. The blind beggar gets Henry, another beggar, to follow Beckert. As Beckert starts peeling an orange with his knife while looking intently at the little girl, Henry writes "M" on his hand with chalk. He then slaps Beckert on the back of the coat and imprints the letter "M" on his back.
Back in Beckert's house, Lohman finds all the evidence he needs to link Beckert to Elsie's murder. At the same time, the little girl with Beckert tells him about the "M" on his back. Beckert looks at the "M" by way of a mirror and starts to panic. He runs leaving the child behind and is pursued by the beggars. They trap him in an office building, then call up Schr�nker.
Schr�nker refuses to call the police about the murderer. He sends men to the office building to find Beckert. They find him in the attic, but not until after they set off the alarm in the building. The criminals beat a hasty retreat with Beckert, but they make the mistake of leaving one man behind. This man gets captured by the police and brought to the station for questioning.
While the cops question the thief, the underworld holds a trial for the child murderer in an abandoned factory. The criminals want to kill him so he will not kill more children. If the police get their hands on him they will just put him into another mental institution and be released later as "harmless" just like before. Beckert's defense lawyer says that he should be handed over to the law because they can help him. Beckert makes a passionate plea for his life, saying that he can not help what he does. The ghosts in his head will not leave him alone unless he kills, which is why he can not stop doing it. Schr�nker, who has killed three men himself, is not moved by his speech. He calls for Beckert's death and all the criminals start to rush at Beckert in a fury. They are all stopped short by the sound of the police as they bust into the factory after finally get the location out of the captured thief. The movie ends with a relieved Beckert being taken away by the police.