So I ended up adding a few statements to diagnose what was happening right before the problematic line 161, and this was helpful:
numel(tagIds)
It prints the number of tags identified in the particular image that is being parsed through. I seemed to get this problem only after multiple poor detections with less than the expected number of tags, and it seemed to happen on the second to last image, which was very odd.
I then went through and individually deleted those files without the full tags detected, and it worked from there. I wouldn't call this a "solution", moreso a workaround, so I'm leaving the question unanswered until someone with better knowledge than I can comment.