Jerome Barberin lives with his wife in a little French town, Chavanon. He usually isn't home, since he works in Paris as a mason. One day he finds a baby boy. The boy wears very fine clothes, so apparently his parents are rich. Barberin offers to take care of the child, hoping to get a good reward. He gives the boy to his wife, and calls him Remi. Afterwards, Barberin gets injured in an accident. He blames his employer and hopes to receive financial compensation in a trial. The trial costs a lot of money, and Barberin tells his wife to sell her cow (her main source of wealth) and to get rid of Remi. She does the former.
When Rémi is eight years old, and this is where the story starts, Barberin comes home unexpectedly. He sees that Rémi is still there and decides to lose no time getting rid of him. The next day Barberin meets a travelling artist in the local pub. His name is Signor Vitalis, and he travels through France with three dogs - Capi, Zerbino and Dolce - and a monkey, Joli-Cœur. Vitalis offers to take care of Rémi and Rémi leaves his childhood home, without even a chance to say goodbye to his foster mother (who would have done anything to prevent the transaction) and starts a journey of the roads of France. It turns out that Vitalis is a kind man, certainly better company than Barberin. Vitalis teaches him to play the harp and to read. Often Rémi is hungry and has no roof over his head; but in the animals, especially in Capi, he gains dear friends, and in Vitalis he finds the father he lacks. Together they travel through France, and they earn a living by giving musical and stage performances.
When they are in Toulouse, a sad incident, which reflects the unjust social structure of 19th Century France, puts Vitalis into jail. It is not easy for a ten-year-old to feed himself and four animals under his care, and they nearly starve, when they meet the "Swan" - a little river ship owned by Mrs. Milligan and her ill son Arthur. They take Rémi in to entertain the sick boy, but soon start seeing a person in Remi, and he becomes part of the family. He learns that Arthur used to have an elder brother, who disappeared before Arthur was born, and Mrs. Milligan's brother-in-law, James, has attempted in vain to find him back. This was advantageous for James Milligan, since, by the English law, he was to inherit all of his brother's fortune if he died childless. This did not work, because soon Arthur was born. After two months Vitalis is released from jail, Remi and the Milligans like to stay together, but Vitalis wants Rémi back, and so they say goodbye. However, Mrs. Milligan judges that Vitalis is a very kind and honest man.
Vitalis tells Rémi that he has done a good choice: one must eat his own bread. But on the way to Paris in a snowstorm Zerbino and Dolce are eaten by wolves in the woods, and Joli-Cœur catches pneumonia. In an attempt to raise money for the doctor, Remi and Vitalis give a performance and Vitalis sings. Remi has never before heard Vitalis sing so beautifully. And not only Rémi is bewildered: a young, and apparently rich lady tells Vitalis that she is amazed to hear his wonderful voice. Vitalis reacts angrily. He explains his skill to the lady by telling that he used to be a singer's servant. The lady explains he has a resemblance to the singer Vitalo Pedrotti from the Scala di Milano who is disparated. He even shows no gladness when the lady gives a gold coin to Capi. They return to Joli-Cœur with the money, but it's too late, Joli-Cœur is dead.
They now continue their journey to Paris. Vitalis decides to leave Rémi with a "padrone" for the winter, while he trains other animals. Another institution of 19th Century France, a "padrone" was a man who kept a group of boys, sold by their poverty-stricken parents, who worked for him. Vitalis brings Remi to a "padrone" he knows - Garofoli. Garofoli isn't home, and Vitalis tells Rémi to wait there, and that he will be back soon. Rémi passes there two horrible hours - waiting for Garofoli and talking to an ill-looking boy, Mattia, who keeps houseworking because Garofoli believes him too stupid and incapable of working outside, but keeping the soup pot locked so that Mattia could not eat from it. When the other boys and Garofoli return, Rémi witnesses how terribly Garofoli abuses those who do not bring home the amount of money required: he beats and starves them...