Add Org dynamic block to insert files as headings

README.org

@@ -3399,6 +3399,94 @@ parameters, which are described further below:
   processing using Org facilities (a feature that is outside Denote's
   purview).
 
+** Org dynamic block to insert Org files as headings
+:PROPERTIES:
+:CUSTOM_ID: h:d6254a12-b762-4096-a5de-66a0d423e204
+:END:
+
+[ Part of {{{development-version}}}. ]
+
+[ IMPORTANT NOTE: This dynamic block only works with Org files,
+  because it has to assume the Org notation in order to insert each
+  file's contents as its own heading. ]
+
+#+findex: denote-org-extras-dblock-insert-files-as-headings
+As a variation of the previously covered block that inserts file
+contents, we have the ~denote-org-extras-dblock-insert-files-as-headings~
+command ([[#h:f15fa143-5036-416f-9bff-1bcabbb03456][Org dynamic block to insert file contents]]). It Turn the
+=#+title= of each file into a top-level heading. Then it increments
+all original headings in the file by one, so that they become
+subheadings of what once was the =#+title=. Similarly, the
+=#+filetags= of each file as tags for the top-level heading
+(what was the =#+title=).
+
+Because of how it is meant to work, this dynamic block only works with
+Org files.
+
+In its simplest form, this dynamic block looks like this:
+
+: #+BEGIN: denote-files-as-headings :regexp "YOUR REGEXP HERE"
+:
+: #+END:
+
+Though when you use the command ~denote-org-extras-dblock-insert-files-as-headings~
+you get all the parameters included:
+
+: #+BEGIN: denote-files-as-headings :regexp "YOUR REGEXP HERE" :excluded-dirs-regexp nil :sort-by-component title :reverse-sort nil :add-links t
+:
+: #+END:
+
+- The =:regexp= parameter is mandatory. Its value is a string,
+  representing a regular expression to match Denote file names. Its
+  value may also be an ~rx~ expression instead of a string, as noted
+  in the previous section ([[#h:50160fae-6515-4d7d-9737-995ad925e64b][Org dynamic blocks to insert links or backlinks]]).
+  Note that you do not need to write an actual regular expression to
+  get meaningful results: even something like =_journal= will work to
+  include all files that have a =journal= keyword.
+
+- The =:excluded-dirs-regexp= is a string that contains a word or
+  regular expression that matches against directory files names
+  to-be-excluded from the results. This has the same meaning as
+  setting the ~denote-excluded-punctuation-regexp~ user option
+  ([[#h:8458f716-f9c2-4888-824b-2bf01cc5850a][Exclude certain directories from all operations]]). The user option
+  has a global effect, which is overridden locally in the dynamic
+  block. When the value of =:excluded-dirs-regexp= is nil (the
+  default), the value of ~denote-excluded-punctuation-regexp~ is used
+  (which is also nil by default, meaning that all directories are
+  included). When the value of =excluded-dirs-regexp= is ~t~ or some
+  other symbol, then the ~denote-excluded-punctuation-regexp~ is
+  ignored altogether. This is useful in the scenario where the user
+  option is set to exclude some directories but the dynamic blocks
+  wants to lift that restriction.
+
+- The =:sort-by-component= parameter is optional. It sorts the files
+  by the given Denote file name component. The value it accepts is an
+  unquoted symbol among =title=, =keywords=, =signature=, =identifier=.
+  When using the command ~denote-org-extras-dblock-insert-files~, this
+  parameter is automatically inserted together with the (=:regexp=
+  parameter) and the user is prompted for a file name component.
+
+- The =:reverse-sort= parameter is optional. It reverses the order in
+  which files appear in. This is meaningful even without the presence
+  of the parameter =:sort-by-component=, though it also combines with
+  it.
+
+- The =:add-links= parameter is optional. When it is set to a ~t~
+  value, all files are inserted as a typographic list and are indented
+  accordingly. The first line in each list item is a link to the file
+  whose contents are inserted in the following lines. When the value
+  is =id-only=, then links are inserted without a description text but
+  only with the identifier of the given file. This has the same
+  meaning as with the ~denote-link~ command and related facilities
+  ([[#h:fc913d54-26c8-4c41-be86-999839e8ad31][Linking notes]]). Remember that Org can fold the items in a
+  typographic list the same way it does with headings. So even long
+  files can be presented in this format without much trouble.
+
+- An optional =:block-name= parameter can be specified with a string
+  value to add a =#+name= to the results. This is useful for further
+  processing using Org facilities (a feature that is outside Denote's
+  purview).
+
 * Sort files by component
 :PROPERTIES:
 :CUSTOM_ID: h:9fe01e63-f34f-4479-8713-f162a5ca865e

denote-org-extras.el

@@ -601,6 +601,115 @@ Used by `org-dblock-update' with PARAMS provided by the dynamic block."
     (when rx (denote-org-extras-dblock-add-files rx separator no-f-m add-links sort reverse excluded-dirs)))
   (join-line)) ; remove trailing empty line
 
+;;;; Insert files as headings
+
+(defun denote-org-extras-dblock--extract-regexp (regexp)
+  "Extract REGEXP from the buffer and trim it of surrounding spaces."
+  (string-trim
+   (save-excursion
+     (re-search-forward regexp nil :no-error)
+     (buffer-substring-no-properties (match-end 0) (line-end-position)))))
+
+(defun denote-org-extras-dblock--get-file-contents-as-heading (file add-links)
+  "Insert the contents of Org FILE, formatting the #+title as a heading.
+With optional ADD-LINKS, make the title link to the original file."
+  (when-let ((_ (denote-file-is-note-p file))
+             (identifier (denote-retrieve-filename-identifier file))
+             (file-type (denote-filetype-heuristics file))
+             (_ (eq file-type 'org)))
+    (with-temp-buffer
+      (let ((beginning-of-contents (point))
+            title
+            tags)
+        (insert-file-contents file)
+        (setq title (denote-org-extras-dblock--extract-regexp (denote--title-key-regexp file-type)))
+        (setq tags (denote-org-extras-dblock--extract-regexp (denote--keywords-key-regexp file-type)))
+        (delete-region (1+ (re-search-forward "^$" nil :no-error 1)) beginning-of-contents)
+        (goto-char beginning-of-contents)
+        (when (and title tags)
+          (if add-links
+              (insert (format "* [[denote:%s][%s]] %s\n\n" identifier title tags))
+            (insert (format "* %s %s\n\n" title tags)))
+          (org-align-tags :all))
+        (while (re-search-forward "^\\(*+?\\) " nil :no-error)
+          (replace-match (format "*%s " "\\1"))))
+      (buffer-string))))
+
+(defun denote-org-extras-dblock-add-files-as-headings (regexp &optional add-links sort-by-component reverse excluded-dirs-regexp)
+  "Insert files matching REGEXP.
+
+If optional ADD-LINKS is non-nil, first insert a link to the file
+and then insert its contents.  In this case, format the contents
+as a typographic list.
+
+If optional SORT-BY-COMPONENT is a symbol among `denote-sort-components',
+sort files matching REGEXP by the corresponding Denote file name
+component.  If the symbol is not among `denote-sort-components',
+fall back to the default identifier-based sorting.
+
+If optional REVERSE is non-nil reverse the sort order.
+
+Optional EXCLUDED-DIRS-REGEXP is the `let' bound value of
+`denote-excluded-directories-regexp'.  When nil, the original value of
+that user option is used."
+  (let* ((denote-excluded-directories-regexp (or excluded-dirs-regexp denote-excluded-directories-regexp))
+         (files (denote-org-extras-dblock--files regexp sort-by-component reverse))
+         (files-contents (mapcar
+                          (lambda (file)
+                            (denote-org-extras-dblock--get-file-contents-as-heading file add-links))
+                          files)))
+    (insert (string-join files-contents))))
+
+;;;###autoload
+(defun denote-org-extras-dblock-insert-files-as-headings (regexp sort-by-component)
+  "Create Org dynamic block to insert Denote Org files matching REGEXP.
+
+Turn the #+title of each file into a top-level heading.  Then increment
+all original headings in the file by one, so that they become
+subheadings of what once was the #+title.
+
+Use the #+filetags of each file as tags for the top-level heading (what
+was the #+title).
+
+Sort the files according to SORT-BY-COMPONENT, which is a symbol
+among `denote-sort-components'.
+
+IMPORTANT NOTE: This dynamic block only works with Org files, because it
+has to assume the Org notation in order to insert each file's contents
+as its own heading."
+  (interactive
+   (list
+    (denote-files-matching-regexp-prompt)
+    (denote-sort-component-prompt))
+   org-mode)
+  (org-create-dblock (list :name "denote-files-as-headings"
+                           :regexp regexp
+                           :excluded-dirs-regexp nil
+                           :sort-by-component sort-by-component
+                           :reverse-sort nil
+                           :add-links nil))
+  (org-update-dblock))
+
+;; NOTE 2024-03-30: This is how the autoload is done in org.el.
+;;;###autoload
+(eval-after-load 'org
+  '(progn
+     (org-dynamic-block-define "denote-files-as-headings" 'denote-org-extras-dblock-insert-files-as-headings)))
+
+;;;###autoload
+(defun org-dblock-write:denote-files-as-headings (params)
+  "Function to update `denote-files' Org Dynamic blocks.
+Used by `org-dblock-update' with PARAMS provided by the dynamic block."
+  (let* ((regexp (plist-get params :regexp))
+         (rx (if (listp regexp) (macroexpand `(rx ,regexp)) regexp))
+         (sort (plist-get params :sort-by-component))
+         (reverse (plist-get params :reverse-sort))
+         (block-name (plist-get params :block-name))
+         (add-links (plist-get params :add-links))
+         (excluded-dirs (plist-get params :excluded-dirs-regexp)))
+    (when block-name (insert "#+name: " block-name "\n"))
+    (when rx (denote-org-extras-dblock-add-files-as-headings rx add-links sort reverse excluded-dirs)))
+  (join-line)) ; remove trailing empty line
 
 (provide 'denote-org-extras)
 ;;; denote-org-extras.el ends here