Would it be practical in your situation to transpose the matrix, so that you were setting columns to 0 instead of rows? With the implementation of sparse(), it is more efficient to change multiple items in one column.
Beyond that: the usual efficiency hint is to find() the non-zero objects of interest, and then use those indices to assign new values.
It would be interesting to benchmark,
J(vec,:) = 0;
compared to
[r,c] = find(J(vec,:));
J(sub2ind(size(J), vec(r), c)) = 0;
compared to
[r,c,s] = find(J(vec,:));
J = J - sparse(vec(r), c, s, size(J,1), size(J,2));
The third of those is locating the non-zero elements of those rows of J, along with the values there, and building a new sparse array that contains only those elements, and then subtracts the two.
