editing HTML files with commands from JSON file

[petermr]

HTML files can be edited using commands in a JSON file. These commands have an xpath selector and a command. Initially only delete, useful for cleaning foreign HTML files created by styling software.

[petermr]

delete commands

Command file:
edit_toplevel.json

{"commands": [
  {"delete": {
    "xpath":  "//noscript",
    "comment":  "no useful content"}
  },
  {"delete": {
    "xpath":  "//script"}
  },
  {"delete": {
    "xpath":  "//meta"}
  },
  {"delete": {
    "xpath":  "//style"}
  },
  {"delete": {
    "xpath":  "//link"}
  },
  {"delete": {
    "xpath":  "//footer"}
  },
  {"delete": {
    "xpath":  "//picture"}
  },
  {"delete": {
    "xpath":  "//div[@class='artistLink']"}
  },
  {"delete": {
    "xpath":  "//div[@class='nav2']"}
  },
  {"delete": {
    "xpath":  "//div[count(*)=0]"}
  },
  {"delete": {
    "xpath":  "//div[img]"}
  },
  {
    "delete": {
      "xpath": "//div[count(*)=0]"}
  },
  {
    "style":
      [
        {"css":
          {
            "selector": "div",
            "value": {
              "border": "solid red 1px",
              "margin": "5px"
            }
          }
        }
      ]
    }
  ]
}

class HtmlEditorTest(AmiAnyTest):
    def test_execute_commands(self):
        editor = HtmlEditor()
        wg = "wg1"
        ar6 = Path(Resources.TEST_RESOURCES_DIR, "ar6")
        IN_WG = Path(ar6, wg)
        OUT_WG = Path(Resources.TEMP_DIR, "ipcc", wg)
        inpath = Path(IN_WG, "toplevel.html")

apply edit commands

        editor.read_html(inpath)
        json_path = Path(ar6, "edit_toplevel.json")
        editor.read_commands(json_path)
        editor.execute_commands()

        outpath = Path(OUT_WG, "edited_toplevel.html")
        HtmlLib.write_html_file(editor.html_elem, outpath, debug=True)

[petermr]

Extended this approach to the de-gatsbification of web pages:

• commands in test/resources/ar6/de_gatsby.json
  Current content:

{"commands": [
  {"delete":
	  {"xpath":
		  [
            "//noscript",
            "//meta",
            "//iframe",
            "//script",
            "//style",
            "//link",
            "//div[not(*) and not(text())]",
            "//footer",
            "//span[@class='arrow-up' or @class='arrow-down' or @class='share-block']",
            "//div[@class='ch-figure-button-cont'",
            "//div[h3='Explore more']",
            "//div[a[button[span='Read more']]]",
            "//div[nav]",
            "//button[.='Expand section']",
            "//div[@class='ref-tooltip' or @class='share-tooltip']",
            "//div[@id='section-tooltip']",
            "//div[div[@class='dropdown']]",
            "//button[span='View']",
            "//div[a[button='Open figure']]"

          ]
       }
     }
   ]
}