There is a very crude brute force method, but it is only practical for a limited range of sizes of the two groups. I suppose first that you want to keep the respective sizes of the two groups unchanged. Let n1 be the number in one group and n2 that in the other group, and let s be the sum of all the ages in the two groups. Use Matlab's 'nchoosek' function to obtain all possible n1 subsets of the n1+n2 total. For each subset find the sum of its elements, s1, and compute abs(s1/n1-(s-s1)/n2) and then choose the minimum of all these absolute differences.
In case you allow the sizes to vary, do the above for all possible values of n1 in 0 < n1 < total_size, and choose the minimum of all these minima.
As I have indicated, I do not recommend the above for very large values of n1+n2.