Prerequisites checklist

• I have read the contributing guidelines.

What is the purpose of this pull request? (put an "X" next to an item)

[ ] Documentation update
[ ] Bug fix (template)
[ ] New rule (template)
[ ] Changes an existing rule (template)
[ ] Add autofix to a rule
[ ] Add a CLI option
[ ] Add something to the core
[x] Other, please explain:

Improve worker count calculation for "auto" concurrency.

refs #20040 (comment)

What changes did you make? (Give an overview)

This pull request makes some improvements to the thread count calculation that is done when the concurrency is "auto".

1. Ignored files are no longer considered in the thread count calculation. Determining if a file is ignored can be done very efficiently now because config arrays are pre-loaded for all files when the files are enumerated.
2. When using the cache with the default "metadata" strategy, cached files with unchanged results are no longer considered in the thread count calculation. This means that only files which need to be read, parsed and processed by the linter will be counted. If all cached results are valid, multithreading will not be used.
3. Increased the AUTO_FILES_PER_WORKER constant from 35 to 50. This constant value affects the thread count calculation when only a few files are linted. The most visible effect is that multithreading will be no longer turned on when linting less than 51 files (this is an empiric value and possibly still too low on some machines). I realized that some of the projects I used in my early tests were biased by ignored files. After repeating the tests with ignored files not counted, the larger value seems more appropriate.

Is there anything you'd like reviewers to focus on?

The time to count files that need processing (excluding ignored and cached files) is printed in a debug line, for example:

  eslint:eslint 351 file(s) to process counted in 18 ms +19ms