A scanned PDF's file size is driven almost entirely by the resolution and compression of the images inside it — page count matters too, but per-page image quality is usually the bigger factor.
Image resolution
Higher resolution scans preserve more fine detail but produce proportionally larger files — for typical text documents, a moderate resolution is usually enough for full legibility without the file size of a print-quality archival scan.
Compression
JPEG-style compression (common inside scanned PDFs) trades a small amount of visual quality for a significant reduction in file size — well-tuned compression is often visually indistinguishable from the uncompressed original at normal viewing sizes.
Color vs. grayscale
A scan that doesn't need color — most text documents — can be converted to grayscale or a high-contrast black-and-white filter, which meaningfully shrinks file size since there's far less color data to store per pixel.
Does merging many scans into one PDF make the file unreasonably large?
It scales roughly linearly with page count, so a 20-page merged document will be noticeably larger than a single scan — reasonable per-page compression keeps even fairly long documents manageable to share by email or messaging apps.