5. Do you define blood relatives as having one common ancestor (male or female)?
Males: A, B, C. Females: 1,2,3
First generation brings up 3 males: A1, B2, C3 and 3 females: 1A, 2B, 3C (simple notation: all males start with a letter and all females with a digit).
Then 1A cannot mate with A and A1 because she has A in common.
And if 1A mates with B, two offsprings are produced: 1AB and B1A. Neither of which can mate with anyone who has A or B or 1 in their system?
If so, then the cap is the number of blood types/colors and there appears to be no infinite solution.
Just a question of when will the process block. At most 5 generations since:
1. After the second mating A,B,C,1,2,3 die. Since mating adds the number of colors offsprings receive, bicolored offsprings can only be produced during the 1st and at most 2nd mating.
2. Everyone involved in the third mating would have at least 2 colors hence each of their offspring will have at least 4 colors.
3. After a fourth mating all possible bicolored offsprings of A,B,C will die, leaving offsprings with 4 or more colors unable to mate with anyone in a fifth mating. All offsprings with exactly 3 colors (if any) may mate with their negative colors but they create 6 colored offsprings.
4. All offsprings with exactly 3 colors (if any) cannot survive after 5th mating since they were born in 3rd mating.
Trying to get an instance with exactly 5 matings is subject to the next question.
6. When you say that all creatures will choose a mating situation, if one exists, where everyone mates, does this only apply to choosing a global optimal solution? Or do they need to "choose" a situation where as many as possible mate? (Forced / Greedy approach).
If the assumption only applies to the global optimal - choose a scenario where all mate, but if at least one can't mate, choose any scenario possible including leaving aside potential mating pairs, then it may be possible to prolong the number of generations. Sorta like a "don't mate, don't die" syndrome.
Which is why I'm assuming you want to assume that "random" also selects the largest number of matings possible.
In which case, trying to prolong the lives of these creatures and dillute the blood as much as possible I can only find 4 matings possible:
Mating #1: A+1, B+2, C+3
Remaining 12: 1 life - A,B,C,1,2,3, 2 lives - A1,B2,C3,1A,2B,3C.
Mating #2 (trying to produce 2 colored offsprings): A+2, B+3, C+1, A1+2B, B2+3C, C3+1A
Remaining 6*2+6 (forefathers die) = 18:
1 life - A1,B2,C3,1A,2B,3C
2 lives - A2, B3, C1, 2A,3B,1C, 12AB, AB12, 23BC, BC23, 13AC, AC13
A1+23BC, B2 + 13AC, C3+AB12, 1A+BC23, 2B+AC13, 3C+12AB - all produce 6*2 colored offsprings who can't mate.
A2+3B, B3+1C, C1+2A - 6 4-colored ones
Remaining 6*2 + 6 + 6*2 = 30
1 life - A2, B3, C1, 2A, 3B, 1C, 12AB, AB12, 23BC, BC23, 13AC, AC13
2 lives - AB23, BC13, AC12, 23AB, 13BC, 12AC, and a bunch of 123ABC and ABC123s who can't mate.
From 18 viable candidates, only the 2-colored once can mate again before they die.
All have >4 colors, there's no one to mate with for a fifth mating.