You can definitely just replace the trays, yes.
Lifters? Umm...probably need to inspect first. Same for the camshaft. Any wear, and both should be replaced. Are you doing the work yourself? If hiring, ask your mechanic for his opinion on tbe wear.
On the Summit Racing camshafts, tbe...