The documentation for "dsp.NCO" states that it uses the same algorithm as the NCO block, as stated here: https://www.mathworks.com/help/dsp/ref/dsp.nco-system-object.html#bsfy7o1-6. The algorithm description can be found here: https://www.mathworks.com/help/dsp/ref/nco.html#mw_51f00334-ea26-4072-9715-49115eae1159.
If dither is disabled during the construction of the "dsp.NCO" object, it appears that "nco" essentially returns parts of the lookup table as the output. With this understanding, you can infer the lookup table values by adjusting the "SamplesPerFrame" of your NCO object and using an appropriate "Phase Increment".
Alternatively, the Signal Processing Toolbox includes an example implementation of the NCO algorithm. You can access and analyze this example to understand the lookup table by using the following command:
openExample('signal/SFDRMeasurementExample')
