Skip to content

Commit

Permalink
Added pelican plugins recommended files
Browse files Browse the repository at this point in the history
- Unit test added
  • Loading branch information
arulrajnet committed Jan 11, 2025
1 parent 4147116 commit bfe3182
Show file tree
Hide file tree
Showing 14 changed files with 546 additions and 58 deletions.
15 changes: 15 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
root = true

[*]
charset = utf-8
end_of_line = lf
indent_size = 4
indent_style = space
insert_final_newline = true
trim_trailing_whitespace = true

[*.py]
max_line_length = 88

[*.yml]
indent_size = 2
2 changes: 2 additions & 0 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
github: justinmayer
liberapay: pelican
100 changes: 100 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
name: build

on: [push, pull_request]

env:
PYTEST_ADDOPTS: "--color=yes"

permissions:
contents: read

jobs:
test:
name: Test - Python ${{ matrix.python-version }}
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]

steps:
- uses: actions/checkout@v4
with:
persist-credentials: false

- name: Set up Python ${{ matrix.python-version }} & PDM
uses: pdm-project/setup-pdm@v4
with:
python-version: ${{ matrix.python-version }}
cache: true
cache-dependency-path: ./pyproject.toml

- name: Install dependencies
run: pdm install

- name: Run tests
run: pdm run invoke tests

lint:
name: Lint
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
persist-credentials: false

- name: Validate links in Markdown files
uses: JustinBeckwith/linkinator-action@v1
with:
retry: true
linksToSkip: "https://pypi.org/project/pelican-statistics/"

- name: Set up Python & PDM
uses: pdm-project/setup-pdm@v4
with:
python-version: "3.10"

- name: Install dependencies
run: pdm install

- name: Run linters
run: pdm run invoke lint --diff

deploy:
name: Deploy
environment: Deployment
needs: [test, lint]
runs-on: ubuntu-latest
if: github.ref=='refs/heads/main' && github.event_name!='pull_request'

permissions:
contents: write
id-token: write

steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Check release
id: check_release
run: |
python -m pip install autopub[github]
autopub check
- name: Publish
if: ${{ steps.check_release.outputs.autopub_release=='true' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
autopub prepare
autopub commit
autopub build
autopub githubrelease
- name: Upload package to PyPI
if: ${{ steps.check_release.outputs.autopub_release=='true' }}
uses: pypa/gh-action-pypi-publish@release/v1
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
.pdm-python
.pdm-build
pdm.lock
.venv
dist
__pycache__
*.pyc
28 changes: 28 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
ci:
autoupdate_schedule: quarterly

# See https://pre-commit.com/hooks.html for info on hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: check-added-large-files
- id: check-ast
- id: check-case-conflict
- id: check-docstring-first
- id: check-merge-conflict
- id: check-toml
- id: check-yaml
- id: debug-statements
- id: detect-private-key
- id: end-of-file-fixer
- id: forbid-new-submodules
- id: trailing-whitespace

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.9.1
hooks:
- id: ruff
- id: ruff-format
args: ["--check"]
9 changes: 9 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Contributing
============

Contributions are welcome and much appreciated. Every little bit helps. You can contribute by improving the documentation, adding missing features, and fixing bugs. You can also help out by reviewing and commenting on [existing issues][].

To start contributing to this plugin, review the [Contributing to Pelican][] documentation, beginning with the **Contributing Code** section.

[existing issues]: https://github.com/pelican-plugins/statistics/issues
[Contributing to Pelican]: https://docs.getpelican.com/en/latest/contribute.html
74 changes: 74 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
Statistics: A Plugin for Pelican
====================================================

[![Build Status](https://img.shields.io/github/actions/workflow/status/pelican-plugins/statistics/main.yml?branch=main)](https://github.com/pelican-plugins/statistics/actions)
[![PyPI Version](https://img.shields.io/pypi/v/pelican-statistics)](https://pypi.org/project/pelican-statistics/)
[![Downloads](https://img.shields.io/pypi/dm/pelican-statistics)](https://pypi.org/project/pelican-statistics/)
![License](https://img.shields.io/pypi/l/pelican-statistics?color=blue)

Pelican plugin that calculates post statistics such as word count, reading ease, and more.

Installation
------------

This plugin can be installed via:

python -m pip install pelican-statistics

As long as you have not explicitly added a `PLUGINS` setting to your Pelican settings file, then the newly-installed plugin should be automatically detected and enabled. Otherwise, you must add `statistics` to your existing `PLUGINS` list. For more information, please see the [How to Use Plugins](https://docs.getpelican.com/en/latest/plugins.html#how-to-use-plugins) documentation.

Usage
-----

This plugin to calculate various statistics about a post and store them in an `article.statistics` dictionary:

- `wc`: how many words
- `read_mins`: how many minutes would it take to read this article, based on 250 wpm (http://en.wikipedia.org/wiki/Words_per_minute#Reading_and_comprehension)
- `word_counts`: frquency count of all the words in the article; can be used for tag/word clouds
- `fi`: Flesch-kincaid Index/ Reading Ease (see: http://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests)
- `fk`: Flesch-kincaid Grade Level

Example:

```python
{
'wc': 2760,
'fi': '65.94',
'fk': '7.65',
'word_counts': Counter({u'to': 98, u'a': 90, u'the': 83, u'of': 50, ...}),
'read_mins': 12
}
```

This allows you to output these values in your templates, like this, for example:

```html
<p title="~{{ article.statistics['wc'] }} words">~{{ article.statistics['read_mins'] }} min read</p>
<ul>
<li>Flesch-kincaid Index/ Reading Ease: {{ article.statistics['fi'] }}</li>
<li>Flesch-kincaid Grade Level: {{ article.statistics['fk'] }}</li>
</ul>
```

The `word_counts` variable is a python `Counter` dictionary and looks something like this, with each unique word and it's frequency:

```python
Counter({u'to': 98, u'a': 90, u'the': 83, u'of': 50, u'karma': 50, .....
```

and can be used to create a tag/word cloud for a post.

Contributing
------------

Contributions are welcome and much appreciated. Every little bit helps. You can contribute by improving the documentation, adding missing features, and fixing bugs. You can also help out by reviewing and commenting on [existing issues][].

To start contributing to this plugin, review the [Contributing to Pelican][] documentation, beginning with the **Contributing Code** section.

[existing issues]: https://github.com/pelican-plugins/statistics/issues
[Contributing to Pelican]: https://docs.getpelican.com/en/latest/contribute.html

License
-------

This project is licensed under the AGPL-3.0 license.
2 changes: 1 addition & 1 deletion pelican/plugins/statistics/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
from .post_stats import *
from .statistics import * # noqa: F403,PGH004,RUF100
13 changes: 10 additions & 3 deletions pelican/plugins/statistics/readability.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,20 @@
# -*- coding: utf-8 -*-

# Adadpted from here: http://acdx.net/calculating-the-flesch-kincaid-level-in-python/
# See here for details: http://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_test
"""
Adadpted from here: http://acdx.net/calculating-the-flesch-kincaid-level-in-python/
See here for details: http://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_test
"""

from __future__ import division
import re


def mean(seq):
"""Return the mean of a sequence."""
return sum(seq) / len(seq)


def syllables(word):
"""Return the number of syllables in a word."""
if len(word) <= 3:
return 1

Expand All @@ -20,6 +23,7 @@ def syllables(word):


def normalize(text):
"""Normalize a text for readability"""
terminators = ".!?:;"
term = re.escape(terminators)
text = re.sub(r"[^%s\sA-Za-z]+" % term, "", text)
Expand All @@ -28,6 +32,7 @@ def normalize(text):


def text_stats(text, wc):
"""Text stats"""
text = normalize(text)
stcs = [s.split(" ") for s in text.split(". ")]
stcs = [s for s in stcs if len(s) >= 2]
Expand All @@ -43,13 +48,15 @@ def text_stats(text, wc):


def flesch_index(stats):
"""Flesch index"""
stcs, words, sbls = stats
if stcs == 0 or words == 0:
return 0
return 206.835 - 1.015 * (words / stcs) - 84.6 * (sbls / words)


def flesch_kincaid_level(stats):
"""Flesch-Kincaid level"""
stcs, words, sbls = stats
if stcs == 0 or words == 0:
return 0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,23 @@
"""

from pelican import signals
from bs4 import BeautifulSoup
import re
from collections import Counter

from .readability import *
from pelican import signals
from pelican.contents import Article, Page
from pelican.generators import ArticlesGenerator, PagesGenerator
from bs4 import BeautifulSoup

from .readability import text_stats, flesch_index, flesch_kincaid_level


def calculate_stats(instance):
"""
Calculate the statistics for a given instance.
"""

if instance._content is not None:
if type(instance) in (Article, Page) and instance._content is not None:
stats = {}
content = instance._content

Expand Down Expand Up @@ -64,8 +70,31 @@ def calculate_stats(instance):
stats['fi'] = "{:.2f}".format(flesch_index(readability_stats))
stats['fk'] = "{:.2f}".format(flesch_kincaid_level(readability_stats))

instance.statistics = stats
# For backward compatibility added the same in `stats` as well
instance.stats = stats


def run_plugin(generators):
"""Run the Statistics plugin."""
for generator in generators:
if isinstance(generator, ArticlesGenerator):
for article in generator.articles:
calculate_stats(article)
for translation in article.translations:
calculate_stats(translation)
elif isinstance(generator, PagesGenerator):
for page in generator.pages:
calculate_stats(page)
for translation in page.translations:
calculate_stats(translation)


def register():
signals.content_object_init.connect(calculate_stats)
"""Register the Statistics plugin."""
try:
signals.all_generators_finalized.connect(run_plugin)
except AttributeError:
# NOTE: This results in #314 so shouldn't really be relied on
# https://github.com/getpelican/pelican-plugins/issues/314
signals.content_object_init.connect(calculate_stats)
Loading

0 comments on commit bfe3182

Please sign in to comment.