Skip to content

Commit

Permalink
feat: add new setting to filter source-paths that are auto generated …
Browse files Browse the repository at this point in the history 
…for example for cljs projects, the default value should be enought for most cases (), replacing old .
  • Loading branch information
@ericdallo
ericdallo committed Jun 22, 2022
1 parent 45b22f5 commit 69a9a5c
Show file tree
Hide file tree
Showing 6 changed files with 47 additions and 47 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
- Fix references and code lens of defrecord/deftype for cljs files. #1055
- Fix clean-ns to move reader conditionals to before normal requires. #1057
- Add new optional linter: [clj-depend](https://github.com/clj-depend/clj-depend) integration. #957
- Add new setting `:source-paths-ignore-regex` to filter source-paths that are auto generated for example for cljs projects, the default value should be enought for most cases (`["resources.*" "target.*"]`), replacing old `ignore-classpath-directories`.

- Editor
- Add support to rename namespace of namespaced keywords like re-frame events/subs. #978
Expand Down
2 changes: 1 addition & 1 deletion docs/all-available-settings.edn
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
{:source-paths #{"src" "test"} ;; auto-resolved for deps.edn, project.clj or bb.edn projects
:source-aliases #{:dev :test}
:project-specs [] ;; Check the default at clojure-lsp.crawler/default-project-specs
:ignore-classpath-directories false
:source-paths-ignore-regex ["resources.*" "target.*"]
:lint-project-files-after-startup? true
:notify-references-on-file-change true
:compute-external-file-changes true
Expand Down
2 changes: 1 addition & 1 deletion docs/settings.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ __You can find all settings and its default values [here](https://github.com/clo
| `:hover` `:arity-on-same-line?` | Whether to keep the arity on the same line of the function on hover, useful for Emacs users. | `false` |
| `:hover` `:clojuredocs` | Whether to get clojuredocs information on hover, the clojuredocs content is cached. | `true` |
| `:hover` `:hide-file-location?` | Whether to show the full filename and path on hover. | `false` |
| `:ignore-classpath-directories` | will not consider clojure files within the directories specified by your classpath. This is needed, for instance, if your build puts artifacts into `resources` or `target` that you want lsp to ignore. | `false` |
| `:source-paths-ignore-regex` | list of regex to filter source-paths. By default, source-paths are retrieved from classpath, and usually the classpath contains folders that are not directly project code or it's auto-generated like cljs `resources` or `target` folders but it's inside your project. Replace old `:ignore-classpath-directories` setting. | `["resources.*" "target.*"]` |
| `:java` `:download-jdk-source?` | Whether to download JDK source from `:java :jdk-source-download-uri` and cache after startup for JDK classes java support. | `false` |
| `:java` `:home-path` | Whether to use this path to find JDK source and cache after startup for JDK classes java support. | nil |
| `:java` `:jdk-source-uri` | URI containing the JDK source to be used. If `:download-jdk-source?` is enabled and URI is *NOT* `file://` then download the source. Check `https://github.com/clojure-lsp/jdk-source`for more sources. | `https://raw.githubusercontent.com/clojure-lsp/jdk-source/main/openjdk-19/reduced/source.zip` |
Expand Down
1 change: 0 additions & 1 deletion docs/troubleshooting.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,6 @@ e.g.,
"classpath-cmd": ["clj", "-Spath"]
}],
"use-metadata-for-privacy?": true,
"ignore-classpath-directories": true
}
}
}
Expand Down
69 changes: 31 additions & 38 deletions lib/src/clojure_lsp/crawler.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,6 @@

(def startup-logger-tag "[Startup]")

(defn ^:private get-cp-entry-type [^java.io.File e]
(cond (.isFile e) :file
(.isDirectory e) :directory
:else :unkown))

(defn ^:private analyze-source-paths! [paths db* file-analyzed-fn]
(let [kondo-result* (future
(shared/logging-time
Expand All @@ -42,36 +37,34 @@

(defn lerp "Linear interpolation" [a b t] (+ a (* (- b a) t)))

(defn ^:private analyze-external-classpath! [root-path source-paths classpath settings progress-token {:keys [db* producer]}]
(let [ignore-directories? (get settings :ignore-classpath-directories)]
(logger/info "Analyzing classpath for project root" root-path)
(when classpath
(let [source-paths-abs (set (map #(shared/relativize-filepath % (str root-path)) source-paths))
external-paths (cond->> (->> classpath
(remove (set source-paths-abs))
(remove (set source-paths)))
ignore-directories? (remove #(let [f (io/file %)] (= :directory (get-cp-entry-type f)))))
{:keys [new-checksums paths-not-on-checksum]} (shared/generate-and-update-analysis-checksums external-paths nil @db*)
batch-update-callback (fn [batch-index batch-count {:keys [total-files files-done]}]
(let [percentage (lerp (lerp 25 70 (/ (dec batch-index) batch-count))
(lerp 25 70 (/ batch-index batch-count))
(/ files-done total-files))]
(producer/publish-progress producer percentage "Analyzing external classpath" progress-token)))
normalization-config {:external? true
:filter-analysis (fn [analysis]
(-> analysis
(dissoc :namespace-usages)
(update :var-definitions #(remove :private %))))}
kondo-result (shared/logging-time
"External classpath paths analyzed, took %s"
(lsp.kondo/run-kondo-on-paths-batch! paths-not-on-checksum normalization-config batch-update-callback db*))]
(swap! db* #(-> %
(update :analysis-checksums merge new-checksums)
(lsp.kondo/db-with-results kondo-result)))
(shared/logging-time
"Manual GC after classpath scan took %s"
(System/gc))
(swap! db* assoc :full-scan-analysis-startup true)))))
(defn ^:private analyze-external-classpath! [root-path source-paths classpath progress-token {:keys [db* producer]}]
(logger/info "Analyzing classpath for project root" root-path)
(when classpath
(let [source-paths-abs (set (map #(shared/relativize-filepath % (str root-path)) source-paths))
external-paths (->> classpath
(remove (set source-paths-abs))
(remove (set source-paths)))
{:keys [new-checksums paths-not-on-checksum]} (shared/generate-and-update-analysis-checksums external-paths nil @db*)
batch-update-callback (fn [batch-index batch-count {:keys [total-files files-done]}]
(let [percentage (lerp (lerp 25 70 (/ (dec batch-index) batch-count))
(lerp 25 70 (/ batch-index batch-count))
(/ files-done total-files))]
(producer/publish-progress producer percentage "Analyzing external classpath" progress-token)))
normalization-config {:external? true
:filter-analysis (fn [analysis]
(-> analysis
(dissoc :namespace-usages)
(update :var-definitions #(remove :private %))))}
kondo-result (shared/logging-time
"External classpath paths analyzed, took %s"
(lsp.kondo/run-kondo-on-paths-batch! paths-not-on-checksum normalization-config batch-update-callback db*))]
(swap! db* #(-> %
(update :analysis-checksums merge new-checksums)
(lsp.kondo/db-with-results kondo-result)))
(shared/logging-time
"Manual GC after classpath scan took %s"
(System/gc))
(swap! db* assoc :full-scan-analysis-startup true))))

(defn ^:private copy-configs-from-classpath! [classpath settings db]
(when (get settings :copy-kondo-configs? true)
Expand Down Expand Up @@ -166,7 +159,7 @@
(let [classpath (:classpath @db*)]
(logger/info startup-logger-tag "Using cached db for project root" root-path)
(swap! db* assoc
:settings (update settings :source-paths (partial source-paths/process-source-paths root-path classpath)))
:settings (update settings :source-paths (partial source-paths/process-source-paths settings root-path classpath)))
(producer/publish-progress producer 15 "Copying kondo configs" progress-token)
(copy-configs-from-classpath! classpath settings @db*))
(do
Expand All @@ -176,13 +169,13 @@
:project-hash project-hash
:kondo-config-hash kondo-config-hash
:classpath classpath
:settings (update settings :source-paths (partial source-paths/process-source-paths root-path classpath)))
:settings (update settings :source-paths (partial source-paths/process-source-paths settings root-path classpath)))

(producer/publish-progress producer 20 "Copying kondo configs" progress-token)
(copy-configs-from-classpath! classpath settings @db*)
(when (= :project-and-deps (:project-analysis-type @db*))
(producer/publish-progress producer 25 "Analyzing external classpath" progress-token)
(analyze-external-classpath! root-path (-> @db* :settings :source-paths) classpath settings progress-token components))
(analyze-external-classpath! root-path (-> @db* :settings :source-paths) classpath progress-token components))
(logger/info "Caching db for next startup...")
(upsert-db-cache! @db*))))
(producer/publish-progress producer (if fast-startup? 15 75) "Resolving config paths" progress-token)
Expand Down
19 changes: 13 additions & 6 deletions lib/src/clojure_lsp/source_paths.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,16 @@
{:source-paths default-source-paths
:origins #{:default}})))

(defn process-source-paths [root-path classpath given-source-paths]
(let [{:keys [origins source-paths classpath-paths]} (resolve-source-paths root-path classpath given-source-paths)]
(when (contains? origins :settings) (logger/info startup-paths-logger-tag "Using given source-paths:" given-source-paths))
(when (contains? origins :classpath) (logger/info startup-paths-logger-tag "Using source-paths from classpath:" classpath-paths))
(when (contains? origins :default) (logger/info startup-paths-logger-tag "Using default source-paths:" default-source-paths))
(mapv #(->> % (shared/to-file root-path) .getCanonicalPath str) (set source-paths))))
(defn process-source-paths [settings root-path classpath given-source-paths]
(let [source-paths-ignore-regex (get settings :source-paths-ignore-regex ["resources.*" "target.*"])
{:keys [origins source-paths]} (resolve-source-paths root-path classpath given-source-paths)
final-source-paths (->> source-paths
set
(remove (fn [source-path]
(let [relative-source-path (shared/relativize-filepath source-path (str root-path))]
(some #(re-matches (re-pattern %) relative-source-path) source-paths-ignore-regex))))
(mapv #(->> % (shared/to-file root-path) .getCanonicalPath str)))]
(when (contains? origins :settings) (logger/info startup-paths-logger-tag "Using given source-paths:" final-source-paths))
(when (contains? origins :classpath) (logger/info startup-paths-logger-tag "Using source-paths from classpath:" final-source-paths))
(when (contains? origins :default) (logger/info startup-paths-logger-tag "Using default source-paths:" final-source-paths))
final-source-paths))

0 comments on commit 69a9a5c

Please sign in to comment.