Bring a large pot of water to a boil. Cook the potatoes, carrots, and beets in the boiling water until cooked but still firm, 15 to 20 minutes for the potatoes, 25 to 30 minutes for the carrots, and 40 minutes for the beets.
Allow each to cool to the touch before further handling.
Remove and discard the skins from the potatoes and beets. Shred the potatoes, carrots, and beets, keeping them separate.
Arrange about half the onion and half the herring fillets in a layer in the bottom of a bowl. Cover the layer with about half of the grated potatoes, then a layer of about half the carrot, and finally about half the beet.
Spread about 3 tablespoons of mayonnaise over the salad; season with salt and pepper. Repeat the layers in the same order to finish building the salad.
Cover with plastic wrap and chill in the refrigerator at least 1 hour. Serve cold.