About this Homelab Blog

gitfeber
2024-08-11 (update: 2024-08-13)
Motivation

Me (gitfeber) needed a way to document all my homelab changes, features and small workarounds/hacks for everything around my self-hosted server infrastructure. The whole structure is based on a Kubernetes cluster running bare metal on Ubuntu 22.04 LTS. I wanted to create this from scratch all by myself and have this running now for over a year. I host everything from websites, gaming server, AI models and even spend a good amount of my computing power to community projects like World Community Grid.
This Blog was mainly created as a documentation platform for myself. Everyone knows this moment after some weeks/months when something isn't working anymore and you self don't even know how you did it "back then"... For these cases I needed a documentation platform where I can document all my things I do on my hardware. After some messing around with solutions on how to create such a platform, this Blog was the end result.
How its made

As you can already imagine, it's again self-hosted on the same server and without any use of common tools for Blogs like WordPress or something else. It's all based on good old raw HTML/CSS and some small JS scripts. But I did not want to write all this raw HTML by myself (how dare you). I wrote a python script that creates Blog posts out of simple Markdown files. With this technique I only have to write the docs once and forever in easy to write and read Markdown syntax. And this is how the "md_html_converter.py" script was created. I also build a binary file from it using PyInstaller (Create Binary From Python Script), you can download it here
image scripting 8 hours meme
Python script code

I'm a big fan of OSS, so of course I'm making my ugly code available for everyone who is interested (at least half of it was created by ChatGPT and Copilot). Have fun laughing ;) I try to update it from time to time, so you always have the latest garbage-code available. Please consider that the HTML templates and classes in the script are made for my specific Blog layout and style!
python
import os
import datetime
import sys
import argparse
import re
import html
from bs4 import BeautifulSoup

# Define global paths
POSTS_DIR = os.path.expanduser("~/Homelab/Kubernetes/nfs-server/data/websites/gitfeber-com/posts/")
INDEX_FILE = os.path.expanduser("~/Homelab/Kubernetes/nfs-server/data/websites/gitfeber-com/index.html")
ROOT_DIR = os.path.expanduser("~/Homelab/Kubernetes/nfs-server/data/websites/gitfeber-com/")

def convert_markdown_to_html(input_file_path):
    global POSTS_DIR
    title_found = False
    title = ''
    toc_content = ""
    html_start = """<!DOCTYPE html>
<html lang="en">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  <!-- meta -->
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="HandheldFriendly" content="True">
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
  <meta property="og:type" content="website">
  <meta property="og:title" content="{title}">
  <meta name="description" content="{title}">

  <!-- math stuff -->
  <script type="text/javascript" id="MathJax-script" async
    src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js">
  </script>

  <!-- Highlight.js -->
  <link rel="stylesheet" href="../css/atom-one-dark.min.css">
  <script src="../js/highlight.min.js"></script>

  <!-- title -->
  <title>{title}</title>
  <link rel="stylesheet" href="../css/style.css">
  <link rel="stylesheet" href="../css/rtl.css">
  <link rel="stylesheet" href="../css/all.min.css">
  <link rel="stylesheet" href="../css/custom_styles.css">
  <link rel="shortcut icon" href="../favicon.ico" type="image/x-icon">
  <link rel="icon" href="../favicon.ico" type="image/x-icon">
  <link rel="apple-touch-icon" sizes="180x180" href="/img/apple-touch-icon.png">

  <script src="../js/smooth-content-links.js"></script>
  <script src="../js/copy-to-clipboard.js"></script>
  <script src="../js/code-highlight.js"></script>
</head>
<body class="max-width mx-auto px3 ltr">
  <div class="toc" id="toc">
    <strong>Table of Contents</strong>
    <ul>
      {toc}
    </ul>
  </div>
  <div class="content index py4">
    <header>
      <!-- Go Back Button -->
      <a href="../index.html" class="go-back-button">← Go Back</a>
    </header>
    <article class="post" itemscope="" itemtype="http://schema.org/BlogPosting">
      <header>
        <h1 class="posttitle" itemprop="name headline" id="main-title">{title}</h1>
        <div class="meta">
          <span class="author" itemprop="author" itemscope="" itemtype="http://schema.org/Person">
            <span itemprop="name">gitfeber</span>
          </span>
          <div class="postdate">
              <time itemprop="datePublished">{creation_date}</time> (update: <time itemprop="dateModified">{update_date}</time>)
          </div>
        </div>
      </header>
      <div class="content" itemprop="articleBody">"""

    html_end = """      </div>
    </article>
    <footer id="footer">
      <div class="footer-left">
        Copyright &copy; 2024 - <a href="https://github.com/gitfeber">gitfeber</a> | <a class="footer-link" href="/About.html">About</a>
      </div>
    </footer>
  </div>
</body></html>"""

    # Read the input file
    article_body_content = ""
    code_snippet = []  # To collect code lines
    in_code_block = False  # State to check if we are in a code block
    in_ordered_list = False  # State to check if we are in an ordered list
    in_unordered_list = False  # State to check if we are in an unordered list
    current_paragraph = []
    in_table = False
    table_headers = []
    table_rows = []

    def convert_markdown_to_html_tags(text):
        # Preserve code blocks (```...```)
        code_blocks = re.findall(r'```(.*?)```', text, re.DOTALL)
        text = re.sub(r'```(.*?)```', 'CODE_BLOCK_PLACEHOLDER', text, flags=re.DOTALL)

        # Convert block math (<div class="math">\[...\]</div>)
        text = re.sub(r'\<span class="math">\(\\)</span>(.+?)\<span class="math">\(\\)</span>', r'<div class="math">\[\1\]</div>', text, flags=re.DOTALL)

        # Convert inline math (<span class="math">\(...\)</span>)
        text = re.sub(r'\<span class="math">\((.+?)\\)</span>', r'<span class="math">\(\1\)</span>', text)

        # Process inline code (single backticks)
        text = re.sub(r'(?<!<samp>)</samp>([^<samp>\n]+)</samp>(?!`)', r'<samp>\1</samp>', text)

        # Restore code blocks
        for block in code_blocks:
            text = text.replace('```{block}```', f'CODE_BLOCK_PLACEHOLDER', 1)

        # Other conversions (bold, italics, etc.)
        text = re.sub(r'\<em>\</em>(.<em>?)\</em>\*', r'<strong>\1</strong>', text)  # Bold
        text = re.sub(r'\<em>(.</em>?)\*', r'<em>\1</em>', text)  # Italics
        text = re.sub(r'<strike class=green-strikethrough><span class=color-normal>(.*?)<span></strike>', r'<strike class=green-strikethrough><span class=color-normal>\1<span></strike>', text)  # Strikethrough
        text = re.sub(r'!\[(.<em>?)\]\((.</em>?)\)', r'<img src="../img/\2" alt="\1">', text)  # Images
        text = re.sub(r'\[(.<em>?)\]\((.</em>?)\)', r'<a class=green-text href="\2">\1</a>', text)  # Links
        text = re.sub(r'\+\+(.*?)\+\+', r'<ins>\1</ins>', text)  # Underline

        return text

    def slugify(text):
        return re.sub(r'[\W_]+', '-', text.lower())

    try:
        with open(input_file_path, 'r', encoding='utf-8') as file:
            for line_number, line in enumerate(file, start=1):
                line = line.rstrip()
                line = convert_markdown_to_html_tags(line)
                if not title_found and line.startswith('# '):
                    title = line.replace('# ', '').strip()
                    title_found = True
                    # Add title to TOC
                    toc_content += f'<li><a href="#main-title">{title}</a></li>\n'
                elif line.startswith('## '):
                    header_text = line.replace('## ', '').strip()
                    header_id = slugify(header_text)
                    toc_content += f'<li><a href="#{header_id}">{header_text}</a></li>\n'
                    if current_paragraph:
                        article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                        current_paragraph = []
                    article_body_content += f'<h2 id="{header_id}">{header_text}</h2>\n'
                elif line.startswith('### '):
                    header_text = line.replace('### ', '').strip()
                    header_id = slugify(header_text)
                    toc_content += f'<li style="margin-left: 20px;"><a href="#{header_id}">{header_text}</a></li>\n'
                    if current_paragraph:
                        article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                        current_paragraph = []
                    article_body_content += f'<h3 id="{header_id}">{header_text}</h3>\n'
                elif line.startswith('#### '):
                    header_text = line.replace('#### ', '').strip()
                    header_id = slugify(header_text)
                    toc_content += f'<li style="margin-left: 40px;"><a href="#{header_id}">{header_text}</a></li>\n'
                    if current_paragraph:
                        article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                        current_paragraph = []
                    article_body_content += f'<h4 id="{header_id}">{header_text}</h4>\n'
                elif line.startswith('##### '):
                    header_text = line.replace('##### ', '').strip()
                    header_id = slugify(header_text)
                    toc_content += f'<li style="margin-left: 60px;"><a href="#{header_id}">{header_text}</a></li>\n'
                    if current_paragraph:
                        article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                        current_paragraph = []
                    article_body_content += f'<h5 id="{header_id}">{header_text}</h5>\n'
                elif line.startswith('###### '):
                    header_text = line.replace('###### ', '').strip()
                    header_id = slugify(header_text)
                    toc_content += f'<li style="margin-left: 80px;"><a href="#{header_id}">{header_text}</a></li>\n'
                    if current_paragraph:
                        article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                        current_paragraph = []
                    article_body_content += f'<h6 id="{header_id}">{header_text}</h6>\n'
                elif re.match(r'^\d+\.\s', line):  # Ordered list item
                    if not in_ordered_list:
                        if current_paragraph:
                            article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                            current_paragraph = []
                        article_body_content += "<ol>\n"
                        in_ordered_list = True
                    if in_unordered_list:
                        article_body_content += "</ul>\n"
                        in_unordered_list = False
                    list_item_text = line[line.index('.') + 1:].strip()
                    article_body_content += f"<li>{list_item_text}</li>\n"
                elif re.match(r'^-\s', line):  # Unordered list item
                    if not in_unordered_list:
                        if current_paragraph:
                            article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                            current_paragraph = []
                        article_body_content += "<ul>\n"
                        in_unordered_list = True
                    if in_ordered_list:
                        article_body_content += "</ol>\n"
                        in_ordered_list = False
                    list_item_text = line[1:].strip()
                    article_body_content += f"<li>{list_item_text}</li>\n"
                elif line == '---':  # Horizontal rule
                    if current_paragraph:
                        article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                        current_paragraph = []
                    article_body_content += '<hr class="green-hr">\n'  # Green horizontal rule
                elif re.match(r'^\|', line):  # Table row
                    if not in_table:
                        if current_paragraph:
                            article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                            current_paragraph = []
                        in_table = True
                    if '---' in line:  # Skip the separator line in Markdown tables
                        continue
                    cells = [cell.strip() for cell in line.split('|') if cell.strip()]
                    if not table_headers:
                        table_headers = cells
                    else:
                        table_rows.append(cells)
                else:
                    if in_table:
                        article_body_content += '<table class="center-table">\n<thead>\n<tr>\n'
                        for header in table_headers:
                            article_body_content += f"<th>{header}</th>\n"
                        article_body_content += "</tr>\n</thead>\n<tbody>\n"
                        for row in table_rows:
                            article_body_content += "<tr>\n"
                            for cell in row:
                                article_body_content += f"<td>{cell}</td>\n"
                            article_body_content += "</tr>\n"
                        article_body_content += "</tbody>\n</table>\n"
                        in_table = False
                        table_headers = []
                        table_rows = []
                    if in_ordered_list:
                        article_body_content += "</ol>\n"
                        in_ordered_list = False
                    if in_unordered_list:
                        article_body_content += "</ul>\n"
                        in_unordered_list = False
                    # Im Codeblock:
                    elif line.startswith('```'):
                        if in_code_block:
                            in_code_block = False
                            code_language = code_block_language if code_block_language else "plain"

                            # HTML-Escaping der Codezeilen
                            code_html = '\n'.join(f'<span class="line">{html.escape(line)}</span>' for line in code_snippet)

                            # Generiere die Zeilennummern und das Code HTML
                            line_numbers = '\n'.join(f'<span class="line-number">{i + 1}</span>' for i in range(len(code_snippet)))

                            # Prüfen, ob es sich um einen "output" Codeblock handelt
                            if code_block_language == "output":
                                # Erstelle den speziellen HTML-Codeblock für Output ohne Syntax-Highlighting
                                article_body_content += f'''
                                <figure class="highlight output">
                                    <figcaption class="code-language">output</figcaption>
                                    <span class="btn-copy tooltipped tooltipped-sw" aria-label="Copy to clipboard!"><i class="far fa-clone"></i></span>
                                    <table>
                                        <tbody>
                                            <tr>
                                                <td class="code">
                                                    <pre><code class="output">{code_html}</code></pre>
                                                </td>
                                            </tr>
                                        </tbody>
                                    </table>
                                </figure>
                                '''
                            else:
                                # Standard-HTML-Codeblock mit Syntax-Highlighting
                                article_body_content += f'''
                                <figure class="highlight {code_language}">
                                    <figcaption class="code-language">{html.escape(code_block_language)}</figcaption>
                                    <span class="btn-copy tooltipped tooltipped-sw" aria-label="Copy to clipboard!"><i class="far fa-clone"></i></span>
                                    <table>
                                        <tbody>
                                            <tr>
                                                <td class="gutter">
                                                    <pre>{line_numbers}</pre>
                                                </td>
                                                <td class="code">
                                                    <pre><code class="language-{html.escape(code_language)}">{code_html}</code></pre>
                                                </td>
                                            </tr>
                                        </tbody>
                                    </table>
                                </figure>
                                '''
                            code_snippet.clear()
                        else:
                            in_code_block = True
                            code_block_language = line[3:].strip()  # Sprache aus dem Markdown extrahieren
                            if current_paragraph:
                                article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                    elif in_code_block:
                        # Füge jede Codezeile zur Liste hinzu und escape HTML-Zeichen
                        code_snippet.append(line)
                    elif line.strip() == '':
                        # Blank line indicates end of paragraph for normal text lines
                        if current_paragraph:
                            article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"
                            current_paragraph = []
                    else:
                        # Normal lines that are not part of a code block
                        current_paragraph.append(line)

        # Check if the document ends with an open code block
        if in_code_block:
            raise ValueError("Document ends with an open code block without closing ```.")

        # Check if the document ends with an open ordered list
        if in_ordered_list:
            article_body_content += "</ol>\n"

        # Check if the document ends with an open unordered list
        if in_unordered_list:
            article_body_content += "</ul>\n"

        # Check if the document ends with an open table
        if in_table:
            article_body_content += '<table class="center-table">\n<thead>\n<tr>\n'
            for header in table_headers:
                article_body_content += f"<th>{header}</th>\n"
            article_body_content += "</tr>\n</thead>\n<tbody>\n"
            for row in table_rows:
                article_body_content += "<tr>\n"
                for cell in row:
                    article_body_content += f"<td>{cell}</td>\n"
                article_body_content += "</tr>\n"
            article_body_content += "</tbody>\n</table>\n"

        # Add any remaining paragraph content
        if current_paragraph:
            article_body_content += f"<p>{' '.join(current_paragraph)}</p>\n"

        # Current date for replacing in the template
        update_date = datetime.datetime.now().strftime('%Y-%m-%d')

        # Determine the output filename and directory
        if os.path.basename(input_file_path) == "About.md":
            output_file_name = "About.html"
            output_directory = ROOT_DIR
        else:
            output_file_name = title.replace(' ', '_') + '.html' if title_found else 'output.html'
            output_directory = POSTS_DIR

        # Construct the full output file path
        output_file_path = os.path.join(output_directory, output_file_name)

        # Extract the original creation date if the file already exists
        if os.path.exists(output_file_path):
            creation_date = extract_creation_date(output_file_path)
        else:
            creation_date = update_date  # Use the update date as the creation date if the file is new

        # Insert the collected data into the HTML template
        full_html = html_start.format(title=title, creation_date=creation_date, update_date=update_date, toc=toc_content) + article_body_content + html_end

        # Write the generated HTML to the output file
        with open(output_file_path, 'w', encoding='utf-8') as file:
            file.write(full_html)

        print(f"File created/updated: {output_file_path}")

    except ValueError as e:
        print(e)
        sys.exit(1)

def extract_creation_date(file_path):
    with open(file_path, 'r', encoding='utf-8') as file:
        content = file.read()
        start_marker = '<time itemprop="datePublished">'
        end_marker = '</time>'
        start_idx = content.find(start_marker)
        if start_idx != -1:
            start_idx += len(start_marker)
            end_idx = content.find(end_marker, start_idx)
            if end_idx != -1:
                date_str = content[start_idx:end_idx]
                date_str = date_str.strip()
                return date_str
    return datetime.datetime.now().strftime('%Y-%m-%d')

def extract_update_date(file_path):
    with open(file_path, 'r', encoding='utf-8') as file:
        content = file.read()
        start_marker = '<time itemprop="dateModified">'
        end_marker = '</time>'
        start_idx = content.find(start_marker)
        if start_idx != -1:
            start_idx += len(start_marker)
            end_idx = content.find(end_marker, start_idx)
            if end_idx != -1:
                date_str = content[start_idx:end_idx]
                date_str = date_str.strip()
                return date_str
    return "Unknown Date"

def create_index_html():
    global POSTS_DIR, INDEX_FILE

    # Template for the start and end of the index.html
    html_index_start = """<!DOCTYPE html>
<html lang="en">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  <!-- meta -->
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="HandheldFriendly" content="True">
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
  <meta property="og:type" content="website">
  <meta property="og:title" content="Homelab Blog">
  <meta name="description" content="A collection of articles and insights about my personal homelab setup">

  <!-- title -->
  <title>Homelab Blog</title>
</head>
<link rel="shortcut icon" href="./favicon.ico" type="image/x-icon">
<link rel="icon" href="./favicon.ico" type="image/x-icon">
<link rel="stylesheet" href="../css/style.css">
<link rel="stylesheet" href="../css/all.min.css">
<link rel="stylesheet" href="../css/rtl.css">
<link rel="stylesheet" href="../css/custom_styles.css">
<link href="/img/apple-touch-icon.png" rel="apple-touch-icon" sizes="180x180"/>
<script src="../js/search-bar.js"></script>
<body class="max-width mx-auto px3 ltr">
  <div class="content index py4">
    <header id="header">
      <a href="https://gitfeber.com">
        <div id="title">
          <h1>Homelab Blog</h1>
        </div>
      </a>
    </header>
    <div id="search-container">
      <input type="text" id="searchInput" placeholder="Search articles..." onkeyup="searchPosts()">
    </div>
  <section id="writing">
    <ul class="post-list" id="postList">
"""

    html_index_end = """
    </ul>
  </section>
  <footer id="footer">
    <div class="footer-left">
      Copyright &copy; 2024 - <a href="https://github.com/gitfeber">gitfeber</a> | <a class="footer-link" href="/About.html">About</a>
    </div>
  </footer>
</div>
</body>
</html>
"""

    # Always initialize the BeautifulSoup object with the new template
    soup = BeautifulSoup(html_index_start + html_index_end, 'html.parser')

    post_list = soup.find('ul', class_='post-list')

    articles = []
    # Generate the new article links
    for filename in os.listdir(POSTS_DIR):
        if filename.endswith(".html") and filename != "index.html":
            file_path = os.path.join(POSTS_DIR, filename)
            link_path = f"./posts/{filename}"
            creation_date_str = extract_creation_date(file_path)
            update_date_str = extract_update_date(file_path)
            title = filename.replace("_", " ").replace(".html", "")

            # Check if the article is already in the index
            existing_item = post_list.find('a', href=link_path)
            if existing_item:
                # Update the date if it has changed
                date_tag = existing_item.find_previous('time', itemprop='dateModified')
                if date_tag and date_tag.text != update_date_str:
                    date_tag.string.replace_with(update_date_str)
            else:
                # Create a new list item for the new article
                new_item = soup.new_tag('li', <em></em>{'class': 'post-item'})
                meta_div = soup.new_tag('div', <em></em>{'class': 'meta'})
                time_tag = soup.new_tag('time', <em></em>{'itemprop': 'dateModified'})
                time_tag.string = update_date_str
                meta_div.append(time_tag)
                new_item.append(meta_div)
                span_tag = soup.new_tag('span')
                link_tag = soup.new_tag('a', href=link_path)
                link_tag.string = title
                span_tag.append(link_tag)
                new_item.append(span_tag)
                articles.append((link_path, update_date_str, str(new_item)))

    # Collect all current articles
    for li in post_list.find_all('li', class_='post-item'):
        link_tag = li.find('a')
        if link_tag:
            link_path = link_tag['href']
            update_date_tag = li.find('time', itemprop='dateModified')
            if update_date_tag:
                update_date_str = update_date_tag.text
                articles.append((link_path, update_date_str, str(li)))

    # Sort articles by update date (newest first)
    articles.sort(key=lambda x: x[1], reverse=True)

    # Clear existing post list and add sorted articles
    post_list.clear()
    for _, _, item in articles:
        post_list.append(BeautifulSoup(item, 'html.parser'))

    # Write the updated HTML to the index.html file
    with open(INDEX_FILE, 'w', encoding='utf-8') as file:
        file.write(str(soup.prettify(formatter="html")))

    print(f"Index file has been created/updated: {INDEX_FILE}")

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Process a blog text file into HTML format and update index.")
    parser.add_argument("input_file", help="The input text file to be processed")
    args = parser.parse_args()

    # Example usage of the script
    input_file_path = args.input_file  # Path to the text file

    if not os.path.isfile(input_file_path):
        print(f"Error: The file '{input_file_path}' does not exist.")
        sys.exit(1)

    convert_markdown_to_html(input_file_path)
    create_index_html()