Can the IDS merely list the prior art references, or do copies of the prior art references need to be submitted?

It depends on which categories of art are included within the IDS.

Copies of listed US patent literature do not need to be submitted along with the IDS since USPTO examiners have access to all US patent literature.

However, copies of listed non-patent literature (NPL) and foreign patent publications do need to be submitted along with the IDS or they will not be considered by USPTO examiners.

Also, if the copies of NPL or foreign patent publications are not in the English language, the applicant must also submit an English translation or a summary of relevancy in the English language in order for the piece of art to be considered by the examiner.