How do you label egg images to calibrate the grading?

Which classes should the labels use?

Use Vigiovo's four classes: clean, dirty, cracked and unverified. A label in another class, such as leaker or fissured, cannot be compared with the system's output. Before labelling any egg, the team has to agree what each class means on its line.

The clean class requires a clean, intact shell. Brazilian regulation uses the same idea for Class A: shell and cuticle "of normal shape, smooth, clean, intact" (RIISPOA, art. 225). A stain you would not accept in Class A should be labelled dirty, even if the egg looks fine at first glance.

A leaker has no class of its own in Vigiovo. Regulation treats eggs with "rompimento da casca e das membranas testáceas", a broken shell and broken shell membranes, as unfit for human consumption (RIISPOA, art. 500, VII). Decide before you start how these eggs enter the set, and record that decision.

How do you write labelling rules that the team will follow?

Write one short rule for each class, in a single sentence, and gather photos that represent it. A vague rule such as "egg with dirt" produces different labels for the same egg. A rule that says exactly what to look at, such as the share of the shell covered by a stain or the length and depth of a line, leaves less room for interpretation.

Follow this order when you build the rule set:

1. Write the rule for each class in one sentence, stating what the labeller should look at.
2. Collect examples of each class and of each borderline case, with the photo next to its label.
3. Record the decision for dirty and cracked eggs and for eggs that fall at the edge of the image.
4. Review the rules with the whole team before labelling the full set.

Keep the rules with the label set. If anyone changes a rule, the images labelled before the change need to be reviewed.

How do you label borderline cases?

A dirty and cracked egg is the most common conflict. For it, Vigiovo applies a tie rule that sends the egg to the more severe class, and the team should use the same rule. If the human labels follow a different logic, the difference you measure will come from the rule, not from the vision. Regulation also leaves no room here: using and washing dirty cracked eggs to make egg products is prohibited (RIISPOA, art. 228).

A defect that only candling reveals, such as a hairline crack or an interior defect, does not show in the photo. Do not label that egg as clean, because the label would state something the image does not show. Leave it out of the image set or record it separately.

An egg cut by the image border, overlapping another egg or with an abnormal shape gets the unverified label. That is the same rule Vigiovo follows, since it gives no grade to those eggs. Keep unverified in the set: forcing one of these eggs to clean or dirty mixes a guess with a result.

How do you make sure two people label the same way?

Ask two trained people to label the same images, each one working alone, without discussing the cases during the work. Then compare the two sets egg by egg and discuss only the disagreements. Every disagreement has a cause: either the rule does not cover the case, or one of the labellers made a mistake. In both cases the written rule changes, and the affected images are labelled again.

Use operator training as the starting point for reading the egg the same way. If a disagreement keeps appearing for the same kind of image, check the light and the framing before concluding that the labeller got it wrong, as explained in lighting for egg inspection.

How many eggs should you label, and how do you set aside a test set?

Label a few hundred eggs from your line, at different times of the shift and on different days, so the set covers the real variation of the conveyor. Set part of them aside as a test set before you adjust any limit, and do not look at that set while you adjust.

Vigiovo always decides the same way for the same image and configuration, so the labels serve to compare configurations, not to teach the system. The step-by-step guide, with what to adjust and how to measure the distance of each decision to its limit, is in calibrating with labelled eggs. The pilot that uses this set is described in how to run a camera inspection pilot.

Frequently asked questions

How many people should label the same images?

At least two, each working alone. Comparing the two sets shows where the written rule is unclear. After the comparison, the disagreements are discussed, the rule is adjusted and the affected images are labelled again, with the same rule for everyone.

Can I label eggs from the photo alone, without candling?

Yes, as long as the label stays within what the photo shows. Visible dirt and cracks appear in the image. Hairline cracks and interior defects do not, so those eggs should be left out of the set or recorded separately, not given a clean label.

Which class does a leaker go in?

Vigiovo has no class of its own for leakers. Decide before labelling how these eggs enter the set, and record the decision. Regulation treats eggs with a broken shell and broken shell membranes as unfit for human consumption (RIISPOA, art. 500, VII).

Sources

1. Como o Vigiovo classifica ovos sem IA / How Vigiovo grades eggs without AI, Vigiovo
2. Decreto nº 9.013/2017 (RIISPOA), Presidência da República (Brasil)