patx/pickledb
update website to be a comprehensive resource, simplify github readme and direct to website, update readme path in pyproject.toml
Commit 9b47045 · patx · 2025-12-10T14:32:17-05:00
Comments
No comments yet.
Diff
diff --git a/README.md b/README.md
index 24dc478..25c04c9 100644
--- a/README.md
+++ b/README.md
@@ -1,505 +1,15 @@
[](https://patx.github.io/pickledb)
-[](https://pepy.tech/projects/pickledb)
-
-## **pickleDB: Your Lightweight, High-Speed Key-Value Store**
-
-`pickleDB` is a lightweight, in-memory key-value store designed for developers who want **simplicity, speed, and reliability** — without sacrificing modern capabilities. BSD 3-Clause License © Harrison Erd.
-
-- 💫 **Blazing Speed**: Backed by the high-performance [orjson](https://pypi.org/project/orjson/) library, pickleDB handles millions of records with ease. Perfect for applications where every millisecond counts
-- 😋 **Ridiculously Easy to Use**: With its minimalist API, pickleDB makes adding, retrieving, and managing your data as simple as writing a Python list. No steep learning curves. No unnecessary complexity.
-- 🔒 **Rock-Solid Reliability**: Your data deserves to be safe. Atomic saves ensure your database remains consistent—even if something goes wrong.
-- 🐍 **Simple Pythonic Flexibility**: Store strings, lists, dictionaries, and more—all with native Python operations. No need to learn special commands. If you know Python, you already know pickleDB.
-- 🙋 **Community & Contributions**: We’re passionate about making pickleDB better every day. Got ideas, feedback, or an issue to report? Let’s connect on [GitHub Issues](https://github.com/patx/pickledb/issues)
-- 💾 **Portable**: Data is stored as standard JSON, human-readable and cross-language friendly.
-- 🕸️ **Async-Ready**: Non-blocking I/O with [aiofiles](https://pypi.org/project/aiofiles/). Works with web frameworks like Starlette, FastAPI, or [MicroPie](https://patx.github.io/micropie).
-- ⚡ **Unified API**: One class, one set of methods - works seamlessly in **both sync and async** environments.
-- 💢 **Limitations**: The entire dataset resides **in memory** while loaded which might be a constraint on systems with limited RAM for extremely large datasets. pickleDB is designed for simplicity, so it may not meet the needs of applications requiring advanced database features. For larger-scale or concurrent applications requiring a more robust, consider [DataSet](https://dataset.readthedocs.io/en/latest/), [Redis](https://redis.io/), [SQLite](https://www.sqlite.org/), or [MongoDB](https://www.mongodb.com/).
-- 📎 **Useful Links**: [GitHub](https://github.com/patx/pickledb) - [PyPI](https://pypi.org/project/pickleDB/) - [Report an Issue/Ask for Help](https://github.com/patx/pickledb/issues) - [Documentation](https://harrisonerd.com/pickledb)
-
-## Getting Started
-
-### Installation
-Install via pip:
-
-```bash
-pip install pickledb
-```
-
-### Synchronous Example
-
-```python
-from pickledb import PickleDB
-
-db = PickleDB("data.json")
-db.load()
-
-db.set("username", "alice")
-db.set("theme", {"color": "blue", "font": "sans-serif"})
-
-print(db.get("username")) # → "alice"
-
-db.save()
-```
-
-### Asynchronous Example
-
-```python
-import asyncio
-from pickledb import PickleDB
-
-async def main():
- async with PickleDB("data.json") as db:
- await db.set("score", 42)
- value = await db.get("score")
- print(value) # → 42
-
-asyncio.run(main())
-```
-
-
-## Core Methods
-
-| Method | Description |
-|--------|--------------|
-| `load()` | Loads the database from disk (async-aware). |
-| `save()` | Atomically saves the database to disk. |
-| `set(key, value)` | Sets or updates a key. |
-| `get(key, default=None)` | Returns the value for a key. |
-| `remove(key)` | Deletes a key if it exists. |
-| `all()` | Returns a list of all keys. |
-| `purge()` | Clears the entire database. |
-
-All of these methods can be used **synchronously or asynchronously** — just `await` them if inside an event loop.
-
-
-## Performance Highlights
-
-pickleDB demonstrates strong performance for handling large-sized datasets:
-
-| Entries | Memory Load Time | Retrieval Time | Save Time |
-|--------------|------------------|----------------|-----------|
-| **1M** | 0.68 sec | 0.64 sec | 0.03 sec |
-| **10M** | 7.48 sec | 7.27 sec | 0.22 sec |
-| **50M** | 43.36 sec | 36.53 sec | 1.09 sec |
-
-Tests were performed on a Dell XPS 9350 running Ubuntu 24.04 using pickleDB's async mode.
-
-
-## User Guide and Examples
-
-### Add or Update Data
-
-You can add or update key-value pairs using the `set()` method:
-
-```python
-# Add a new key-value pair
-db.set('username', 'admin')
-
-# Or shorthand
-db['username'] = 'admin'
-
-# Update an existing key-value pair
-db.set('username', 'superadmin')
-print(db.get('username')) # Output: 'superadmin'
-```
-
-Keys are automatically converted to strings, and values can be any **JSON-serializable** object.
-
-### Retrieve Values
-
-You can retrieve a keys value using the `get()` method:
-
-```python
-# Get the value for a key
-print(db.get('username')) # Output: 'superadmin'
-
-# Using Python syntax sugar
-db['username'] # Output: 'superadmin'
-
-# Attempt to retrieve a non-existent key
-print(db.get('nonexistent')) # Output: None
-```
-
-### List All Keys
-
-You can get a list of all the keys currently in the database using the `all()` method:
-
-```python
-db.set('item1', 'value1')
-db.set('item2', 'value2')
-
-print(db.all()) # Output: ['username', 'item1', 'item2']
-```
-
-*Note:* This method shows all keys currently loaded, it does **not** guarantee they are persisted to the disk (yet).
-
-### Remove Keys
-
-To remove a key from the database use the `remove()` method:
-
-```python
-db.remove('item1')
-print(db.all()) # Output: ['username', 'item2']
-```
-
-### Purge the Database
-
-To remove all keys and their values from the database use the `purge()` method:
-
-```python
-db.purge()
-print(db.all()) # Output: []
-```
-
-### Saving Data
-
-**pickleDB does not auto-save by default** for performance reasons. To persist data, call `save()` manually or use a context manager:
-
-```python
-db.save() # Output: True
-
-# Context manager example
-with db:
- db.set('foo', 'bar')
- db.set('hello', 'world')
-# Automatically saves when exiting the context
-```
-
-*Note:* All the above methods work/display on the in-memory database. To persist any of the above methods actions use must call the `save()` method or use a context manager, as stated above.
-
-### Asynchronous Usage
-
-pickleDB 1.4 uses a **single unified class** for both synchronous and asynchronous contexts.
+[pickleDB](https://patx.github.io/pickledb) is a fast, easy to use, in-memory Python
+key-value store with first class asynchronous support. It is built with the `orjson`
+module for extremely high performance and was originally inspired by Redis. It is
+licensed under the BSD three-clause license. [Check out the website](https://patx.guthub.io/pickledb)
+for installation instructions, API docs, advanced examples, benchmarks, and more.
```python
-import asyncio
from pickledb import PickleDB
-async def main():
- async with PickleDB('data.json') as db:
- await db.set('score', 42)
- print(await db.get('score')) # Output: 42
-
-asyncio.run(main())
-```
-
-Just `await` any method when inside an async function/event-loop.
-
-### Store and Retrieve Complex Data
-
-```python
-# Store a dictionary
-db.set('user', {'name': 'Alice', 'age': 30, 'city': 'Wonderland'})
-
-# Retrieve and modify
-user = db.get('user')
-user['age'] += 1
-db.set('user', user)
-
-print(db.get('user'))
-# Output: {'name': 'Alice', 'age': 31, 'city': 'Wonderland'}
-```
-
-### Use Lists for Dynamic Data
-
-```python
-db.set('tasks', ['Write code', 'Test app', 'Deploy'])
-
-tasks = db.get('tasks')
-tasks.append('Celebrate')
-db.set('tasks', tasks)
-
-print(db.get('tasks'))
-# Output: ['Write code', 'Test app', 'Deploy', 'Celebrate']
-```
-
-### Advanced Key Search
-
-You can filter keys dynamically using Python list comprehensions:
-
-```python
-def get_keys_with_match(db_instance, match):
- return [key for key in db_instance.all() if match in key]
-
-db.set('apple', 1)
-db.set('apricot', 2)
-db.set('banana', 3)
-
-print(get_keys_with_match(db, 'ap'))
-# Output: ['apple', 'apricot']
-```
-
-### Namespaces
-
-Simulate namespaces using prefixes:
-
-```python
-db.set('user:1', {'name': 'Alice'})
-db.set('user:2', {'name': 'Bob'})
-
-def get_namespace_keys(db_instance, namespace):
- return [key for key in db_instance.all() if key.startswith(f"{namespace}:")]
-
-print(get_namespace_keys(db, 'user'))
-# Output: ['user:1', 'user:2']
-```
-
-### Key Expiration (TTL)
-
-pickleDB doesn’t include TTL natively, but you can simulate it:
-
-```python
-import time
-
-def set_with_expiry(db, key, value, ttl):
- db.set(key, {'value': value, 'expires_at': time.time() + ttl})
-
-def get_if_not_expired(db, key):
- data = db.get(key)
- if data and time.time() < data['expires_at']:
- return data['value']
- db.remove(key)
- return None
-
-set_with_expiry(db, 'session', 'active', ttl=5)
-time.sleep(3)
-print(get_if_not_expired(db, 'session')) # 'active'
-time.sleep(3)
-print(get_if_not_expired(db, 'session')) # None
+async with PickleDB("example.json") as db:
+ await db["key"] = "value"
```
-### Encrypted Storage
-
-```python
-from cryptography.fernet import Fernet
-
-key = Fernet.generate_key()
-cipher = Fernet(key)
-
-encrypted = cipher.encrypt(b"My secret data")
-db.set('secure', encrypted)
-
-decrypted = cipher.decrypt(db.get('secure'))
-print(decrypted.decode()) # Output: My secret data
-```
-
-### Batch Operations
-
-```python
-def batch_set(db, items):
- for key, value in items.items():
- db.set(key, value)
-
-batch_set(db, {'k1': 'v1', 'k2': 'v2', 'k3': 'v3'})
-print(db.all())
-
-def batch_delete(db, keys):
- for key in keys:
- db.remove(key)
-
-batch_delete(db, ['k1', 'k2'])
-print(db.all())
-```
-
-### Key Pattern Matching
-
-```python
-import re
-
-def get_keys_by_pattern(db, pattern):
- regex = re.compile(pattern)
- return [key for key in db.all() if regex.search(key)]
-
-db.set('user:1', {'name': 'Alice'})
-db.set('user:2', {'name': 'Bob'})
-db.set('admin:1', {'name': 'Charlie'})
-
-print(get_keys_by_pattern(db, r'user:\d'))
-# Output: ['user:1', 'user:2']
-```
-
-### Signal Handling for Graceful Shutdowns
-
-```python
-import signal, sys
-from pickledb import PickleDB
-
-db = PickleDB('data.json')
-
-signal.signal(signal.SIGINT, lambda s, f: (db.save(), sys.exit(0)))
-signal.signal(signal.SIGTERM, lambda s, f: (db.save(), sys.exit(0)))
-
-db.set('key1', 'value1')
-print("Running... Press Ctrl+C to save and exit.")
-while True:
- pass
-```
-
-### Using pickleDB with Web Frameworks
-
-Example using [MicroPie](https://patx.github.io/micropie):
-
-```python
-from uuid import uuid4
-from micropie import App
-from pickledb import PickleDB
-
-db = PickleDB('pastes.json')
-
-class Root(App):
- async def index(self, paste_content=None):
- if self.request.method == "POST":
- pid = str(uuid4())
- await db.set(pid, paste)
- await db.save()
- return self._redirect(f'/paste/{pid}')
- return await self._render_template('index.html')
-
- async def paste(self, paste_id):
- paste = await db.get(paste_id)
- return await self._render_template('paste.html', paste_id=paste_id, paste_content=paste)
-
-app = Root()
-```
-
-## Core API Reference
-
-### Class: `PickleDB`
-
-```python
-class PickleDB(location: str)
-```
-
-A lightweight, JSON-backed key-value database. All data is kept in memory while loaded and written atomically to disk on `save()`.
-
-#### Parameters
-| Name | Type | Description |
-|------|-------|-------------|
-| `location` | `str` | Path to the JSON file backing the database. Tilde (`~`) is expanded. |
-
-
-### Context Manager Support
-
-#### Synchronous
-```python
-with PickleDB("data.json") as db:
- db.set("foo", "bar")
-```
-
-#### Asynchronous
-```python
-async with PickleDB("data.json") as db:
- await db.set("foo", "bar")
-```
-
-On successful exit, the DB is automatically saved.
-
-
-### Method Reference
-
-#### `load()`
-
-```python
-load() -> None
-await load() -> None
-```
-
-Loads the database into memory from disk if the file exists and contains valid JSON. Creates an empty database otherwise.
-
-
-#### `save()`
-
-```python
-save() -> bool
-await save() -> bool
-```
-
-Writes the in-memory database to disk.
-
-Returns `True` on success.
-
-Notes:
-- Uses a temporary file + `os.replace` for durability.
-- Automatically called on successful context-manager exit.
-
-#### `set()`
-
-```python
-set(key: str, value: Any) -> bool
-await set(key: str, value: Any) -> bool
-```
-
-Sets or updates a value for a key.
-`key` is coerced to `str`, and `value` must be JSON-serializable.
-
-Returns `True`.
-
-##### Syntax Sugar
-```python
-db["username"] = "alice"
-```
-
-#### `get()`
-
-```python
-get(key: str, default: Any = None) -> Any
-await get(key: str, default: Any = None) -> Any
-```
-
-Retrieves a value by key.
-
-Returns:
-- stored value
-- `default` if key does not exist
-
-##### Syntax Sugar
-```python
-value = db["username"]
-```
-
-
-#### `remove()`
-
-```python
-remove(key: str) -> bool
-await remove(key: str) -> bool
-```
-
-Removes a key.
-
-Returns:
-- `True` if removed
-- `False` if not found
-
-#### `all()`
-
-```python
-all() -> list[str]
-await all() -> list[str]
-```
-
-Returns a list of all keys currently in memory.
-
-#### `purge()`
-
-```python
-purge() -> bool
-await purge() -> bool
-```
-
-Clears the entire in-memory database.
-
-Returns:
-- `True`
-
-### Synchronous vs Asynchronous Behavior
-
-All public methods of `PickleDB` work in both runtimes:
-
-| Environment | Usage |
-|-------------|--------|
-| Synchronous | `db.load()` |
-| Asynchronous | `await db.load()` |
-
-Internally, the library detects whether it is inside an async event loop.
diff --git a/bench.py b/bench.py
deleted file mode 100644
index 10bc9ca..0000000
--- a/bench.py
+++ /dev/null
@@ -1,128 +0,0 @@
-#!/usr/bin/env python3
-import argparse
-import asyncio
-import os
-import time
-from pathlib import Path
-
-from pickledb import PickleDB
-
-
-def human_bytes(n):
- """Return human-readable file size."""
- for unit in ["B", "KB", "MB", "GB"]:
- if n < 1024:
- return f"{n:.2f} {unit}"
- n /= 1024
- return f"{n:.2f} TB"
-
-
-# ------------------------------------------------------------------------------
-# SYNC BENCHMARKS
-# ------------------------------------------------------------------------------
-
-def bench_sync(db_path: Path, count: int):
- print(f"\n=== Sync Benchmark ({count:,} keys) ===")
- db = PickleDB(str(db_path))
-
- # SET
- t0 = time.perf_counter()
- for i in range(count):
- db.set(f"key_{i}", i)
- t1 = time.perf_counter()
-
- # GET
- for i in range(count):
- db.get(f"key_{i}")
- t2 = time.perf_counter()
-
- # SAVE
- db.save()
- t3 = time.perf_counter()
-
- set_rate = count / (t1 - t0)
- get_rate = count / (t2 - t1)
- save_time = t3 - t2
- file_size = human_bytes(os.path.getsize(db_path))
-
- print(f"SET: {set_rate:,.0f} ops/sec ({t1 - t0:.2f}s)")
- print(f"GET: {get_rate:,.0f} ops/sec ({t2 - t1:.2f}s)")
- print(f"SAVE: ({save_time:.2f}s)")
- print(f"File size: {file_size}")
-
-
-# ------------------------------------------------------------------------------
-# ASYNC BENCHMARKS
-# ------------------------------------------------------------------------------
-
-async def bench_async(db_path: Path, count: int):
- print(f"\n=== Async Benchmark ({count:,} keys) ===")
- db = PickleDB(str(db_path))
-
- # SET
- t0 = time.perf_counter()
- for i in range(count):
- await db.set(f"key_{i}", i)
- t1 = time.perf_counter()
-
- # GET
- for i in range(count):
- await db.get(f"key_{i}")
- t2 = time.perf_counter()
-
- # SAVE
- await db.save()
- t3 = time.perf_counter()
-
- set_rate = count / (t1 - t0)
- get_rate = count / (t2 - t1)
- save_time = t3 - t2
- file_size = human_bytes(os.path.getsize(db_path))
-
- print(f"SET: {set_rate:,.0f} ops/sec ({t1 - t0:.2f}s)")
- print(f"GET: {get_rate:,.0f} ops/sec ({t2 - t1:.2f}s)")
- print(f"SAVE: ({save_time:.2f}s)")
- print(f"File size: {file_size}")
-
-
-# ------------------------------------------------------------------------------
-# CLI ENTRY
-# ------------------------------------------------------------------------------
-
-def main():
- parser = argparse.ArgumentParser(description="PickleDB performance benchmark.")
- parser.add_argument(
- "-n", "--count",
- type=int,
- default=100_000,
- help="Number of key-value pairs (default: 100k)"
- )
- parser.add_argument(
- "--async",
- action="store_true",
- dest="async_mode",
- help="Run async version instead of sync"
- )
- parser.add_argument(
- "--db",
- type=str,
- default="benchmark_db.json",
- help="Database filename (default: benchmark_db.json)"
- )
-
- args = parser.parse_args()
- db_path = Path(args.db)
-
- # Remove file if exists
- if db_path.exists():
- db_path.unlink()
-
- if args.async_mode:
- asyncio.run(bench_async(db_path, args.count))
- else:
- bench_sync(db_path, args.count)
-
-
-if __name__ == "__main__":
- main()
-
diff --git a/docs/README.md b/docs/README.md
deleted file mode 100644
index 2293433..0000000
--- a/docs/README.md
+++ /dev/null
@@ -1,27 +0,0 @@
-[](https://patx.github.io/pickledb)
-
-[pickleDB](https://patx.github.io/pickledb) is a fast, easy to use, in-memory Python
-key-value store with asynchronous support. It is built with the `orjson` module for
-extremely high performance and was originally inspired by Redis. It is licensed under
-the BSD three-clause license.
-
-### pickleDB is easy
-
-```python
->>> from pickledb import PickleDB
-
->>> db = PickleDB('example.json')
-
->>> db.set('key', 'value')
-True
-
->>> db.get('key')
-'value'
-```
-
-### Useful links
-
-- **Homepage**: [patx.github.io/pickledb](https://patx.github.io/pickledb)
-- **GitHub**: [github.com/patx/pickledb](https://github.com/patx/pickledb)
-
-
diff --git a/docs/index.html b/docs/index.html
index b39cfc6..87eb3ce 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -1,161 +1,648 @@
<!DOCTYPE html>
<html lang="en">
<head>
- <meta charset="UTF-8">
- <meta name="viewport" content="width=device-width, initial-scale=1.0">
- <link rel="shortcut icon" href="https://pythonhosted.org/pickleDB/favicon.ico" type="image/x-icon">
- <link rel="icon" href="https://pythonhosted.org/pickleDB/favicon.ico" type="image/x-icon">
- <link href="https://fonts.googleapis.com/css2?family=Poppins:wght@300;400;600&display=swap" rel="stylesheet">
- <title>pickleDB - Simple Key-Value Database</title>
- <style>
- body {
- font-family: 'Poppins', sans-serif;
- margin: 0;
- padding: 0;
-
- color: #333;
- }
- .container {
- max-width: 800px;
- margin: auto;
- margin-top: auto;
- margin-bottom: auto;
- background: #fff;
- padding: 40px;
- border-radius: 15px;
- text-align: center;
- }
- h1, h2 {
- font-weight: 600;
- }
- h2 {
- color: #008000;
- text-align: left;
- }
- p {
- font-size: 18px;
- line-height: 1.6;
- color: #555;
- }
- a {
- color: #008000;
- text-decoration: none;
- font-weight: 600;
- transition: color 0.3s ease;
- }
- a:hover {
- color: #008000;
- border-bottom: 1px dotted #008000;
- }
- pre {
- background: #f5f2f0;
- padding: 20px;
- border-radius: 10px;
- text-align: left;
- overflow-x: auto;
- font-size: 1.1em;
- }
- .logo img {
- max-width: 600px;
- }
- span.c2 {
- color: #8F5902;
- }
- span.c9 {
- color: #008000;
- }
- .github-banner img {
- position: fixed;
- top: 0;
- right: 0;
- border: 0;
- }
- .button-container {
- display: flex;
- justify-content: space-between;
- gap: 10px;
- align-items: center;
- }
- .button {
- display: flex; /* Use flexbox for proper vertical centering */
- align-items: center; /* Vertically center the text */
- justify-content: center; /* Horizontally center the text */
- background: #008000;
- color: #fff;
- padding: 0 30px; /* Only horizontal padding */
- height: 60px; /* Set a fixed height for both buttons */
- border-radius: 30px;
- font-size: 18px;
- font-weight: 600;
- text-align: center;
- width: 32%; /* Ensure both buttons take up equal width */
- transition: background 0.3s ease;
- }
- .button:hover {
- background: white;
- border: 2px solid #008000;
- color: #008000;
- }
- .left-button {
- margin-right: 10px;
- }
- .right-button {
- margin-left: 10px;
- }
- .middle-button {
- margin: 0 10px;
- }
- @media (max-width: 768px) {
- .container {
- width: 100%;
- padding: 20px;
- }
- .logo img {
- max-width: 100%;
- }
- .button {
- max-width: 90%;
- text-align: center;
- align-items: center
- }
- .github-banner {
- display: none;
- }
- }
- </style>
+ <meta charset="UTF-8">
+ <title>pickleDB: Your Lightweight, High-Speed Key-Value Store</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <style>
+ body {
+ font-family: system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
+ line-height: 1.6;
+ margin: 0;
+ padding: 2rem 1rem 4rem;
+ max-width: 900px;
+ margin-inline: auto;
+ background: #fafafa;
+ color: #111827;
+ }
+ img {
+ max-width: 100%;
+ height: auto;
+ }
+ a {
+ color: #2563eb;
+ text-decoration: none;
+ }
+ a:hover {
+ text-decoration: underline;
+ }
+ h1, h2, h3 {
+ color: #111827;
+ }
+ h1 {
+ font-size: 2rem;
+ margin-top: 1.5rem;
+ }
+ h2 {
+ font-size: 1.5rem;
+ margin-top: 2rem;
+ }
+ h3 {
+ font-size: 1.25rem;
+ margin-top: 1.5rem;
+ }
+ code {
+ font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace;
+ background: #f3f4f6;
+ padding: 0.15em 0.35em;
+ border-radius: 4px;
+ font-size: 0.95em;
+ }
+ pre {
+ background: #111827;
+ color: #e5e7eb;
+ padding: 1rem;
+ border-radius: 8px;
+ overflow: auto;
+ font-size: 0.9rem;
+ }
+ pre code {
+ background: transparent;
+ padding: 0;
+ }
+ ul {
+ padding-left: 1.25rem;
+ }
+ table {
+ border-collapse: collapse;
+ width: 100%;
+ margin: 1rem 0;
+ font-size: 0.95rem;
+ }
+ th, td {
+ border: 1px solid #e5e7eb;
+ padding: 0.5rem 0.75rem;
+ text-align: left;
+ }
+ thead {
+ background: #f3f4f6;
+ font-weight: 600;
+ }
+ tbody tr:nth-child(even) {
+ background: #f9fafb;
+ }
+ .badge-links {
+ display: flex;
+ flex-wrap: wrap;
+ gap: 0.75rem;
+ align-items: center;
+ margin-bottom: 1.5rem;
+ }
+ .small-note {
+ font-size: 0.85rem;
+ color: #6b7280;
+ }
+ </style>
</head>
<body>
- <div class="container">
- <div class="logo">
- <img src="logo.png" alt="pickleDB logo">
- </div>
- <p><strong>pickleDB is a fast, easy to use, in-memory Python key-value store with asynchronous support.</strong>
- It is built with the <a href="https://pypi.org/project/orjson/">orjson</a> module for extremely high performance and
- was originally inspired by <a href="http://redis.io/">Redis</a>. It is licensed under the BSD three-clause license. View the
- source code, complete user guide, examples and API documentation on <a href="https://github.com/patx/pickledb">GitHub</a>.</p>
-
- <h2 style="margin-top:50px;">pickleDB is Fun</h2>
- <pre><code>
-<span class="c2">>>></span> <span class="c9">from</span> pickledb <span class="c9">import</span> PickleDB
-
-<span class="c2">>>></span> db = PickleDB(<span class="c9">'example.json'</span>)
-
-<span class="c2">>>></span> db.set(<span class="c9">'key', 'value'</span>)
-True
-
-<span class="c2">>>></span> db.get(<span class="c9">'key'</span>)
-'value'
- </code></pre>
-
- <h2 style="margin-top:50px;"> And Easy to Install</h2>
- <pre><code><span class="c2">$</span> pip install <span class="c9">pickledb</span></code></pre>
- </p>
- </div>
- <div class="github-banner">
- <a href="https://github.com/patx/pickledb">
- <img style="position: fixed; top: 0; right: 0; border: 0;" src="https://github.blog/wp-content/uploads/2008/12/forkme_right_green_007200.png" alt="Fork me on GitHub">
- </a>
- </div>
+
+ <div class="badge-links">
+ <a href="https://patx.github.io/pickledb">
+ <img src="https://patx.github.io/pickledb/logo.png" alt="pickleDB Logo">
+ </a>
+
+ <a href="https://pepy.tech/projects/pickledb">
+ <img src="https://static.pepy.tech/badge/pickledb" alt="PyPI Downloads">
+ </a>
+ </div>
+
+ <h1><strong>pickleDB: Your Lightweight, High-Speed Key-Value Store</strong></h1>
+
+ <p><code>pickleDB</code> is a lightweight, in-memory key-value store designed for developers who want <strong>simplicity, speed, and reliability</strong> — without sacrificing modern capabilities. BSD 3-Clause License © Harrison Erd.</p>
+
+ <ul>
+ <li>💫 <strong>Blazing Speed</strong>: Backed by the high-performance <a href="https://pypi.org/project/orjson/">orjson</a> library, pickleDB handles millions of records with ease. Perfect for applications where every millisecond counts.</li>
+ <li>😋 <strong>Ridiculously Easy to Use</strong>: With its minimalist API, pickleDB makes adding, retrieving, and managing your data as simple as writing a Python list. No steep learning curves. No unnecessary complexity.</li>
+ <li>🔒 <strong>Rock-Solid Reliability</strong>: Your data deserves to be safe. Atomic saves ensure your database remains consistent—even if something goes wrong.</li>
+ <li>🐍 <strong>Simple Pythonic Flexibility</strong>: Store strings, lists, dictionaries, and more—all with native Python operations. No need to learn special commands. If you know Python, you already know pickleDB.</li>
+ <li>🙋 <strong>Community & Contributions</strong>: We’re passionate about making pickleDB better every day. Got ideas, feedback, or an issue to report? Let’s connect on <a href="https://github.com/patx/pickledb/issues">GitHub Issues</a>.</li>
+ <li>💾 <strong>Portable</strong>: Data is stored as standard JSON, human-readable and cross-language friendly.</li>
+ <li>🕸️ <strong>Async-Ready</strong>: Non-blocking I/O with <a href="https://pypi.org/project/aiofiles/">aiofiles</a>. Works with web frameworks like Starlette, FastAPI, or <a href="https://patx.github.io/micropie">MicroPie</a>.</li>
+ <li>⚡ <strong>Unified API</strong>: One class, one set of methods - works seamlessly in <strong>both sync and async</strong> environments.</li>
+ <li>💢 <strong>Limitations</strong>: The entire dataset resides <strong>in memory</strong> while loaded which might be a constraint on systems with limited RAM for extremely large datasets. pickleDB is designed for simplicity, so it may not meet the needs of applications requiring advanced database features. For larger-scale or concurrent applications requiring a more robust solution, consider <a href="https://dataset.readthedocs.io/en/latest/">DataSet</a>, <a href="https://redis.io/">Redis</a>, <a href="https://www.sqlite.org/">SQLite</a>, or <a href="https://www.mongodb.com/">MongoDB</a>.</li>
+ <li>📎 <strong>Useful Links</strong>:
+ <a href="https://github.com/patx/pickledb">GitHub</a> -
+ <a href="https://pypi.org/project/pickleDB/">PyPI</a> -
+ <a href="https://github.com/patx/pickledb/issues">Report an Issue/Ask for Help</a> -
+ <a href="https://harrisonerd.com/pickledb">Documentation</a>
+ </li>
+ </ul>
+
+ <h2>Getting Started</h2>
+
+ <h3>Installation</h3>
+ <p>Install via pip:</p>
+ <pre><code class="language-bash">pip install pickledb
+</code></pre>
+
+ <h3>Synchronous Example</h3>
+
+ <pre><code class="language-python">from pickledb import PickleDB
+
+db = PickleDB("data.json")
+db.load()
+
+db.set("username", "alice")
+db.set("theme", {"color": "blue", "font": "sans-serif"})
+
+print(db.get("username")) # → "alice"
+
+db.save()
+</code></pre>
+
+ <h3>Asynchronous Example</h3>
+
+ <pre><code class="language-python">import asyncio
+from pickledb import PickleDB
+
+async def main():
+ async with PickleDB("data.json") as db:
+ await db.set("score", 42)
+ value = await db.get("score")
+ print(value) # → 42
+
+asyncio.run(main())
+</code></pre>
+
+ <h2>Core Methods</h2>
+
+ <table>
+ <thead>
+ <tr>
+ <th>Method</th>
+ <th>Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><code>load()</code></td>
+ <td>Loads the database from disk (async-aware).</td>
+ </tr>
+ <tr>
+ <td><code>save()</code></td>
+ <td>Atomically saves the database to disk.</td>
+ </tr>
+ <tr>
+ <td><code>set(key, value)</code></td>
+ <td>Sets or updates a key.</td>
+ </tr>
+ <tr>
+ <td><code>get(key, default=None)</code></td>
+ <td>Returns the value for a key.</td>
+ </tr>
+ <tr>
+ <td><code>remove(key)</code></td>
+ <td>Deletes a key if it exists.</td>
+ </tr>
+ <tr>
+ <td><code>all()</code></td>
+ <td>Returns a list of all keys.</td>
+ </tr>
+ <tr>
+ <td><code>purge()</code></td>
+ <td>Clears the entire database.</td>
+ </tr>
+ </tbody>
+ </table>
+
+ <p>All of these methods can be used <strong>synchronously or asynchronously</strong> — just <code>await</code> them if inside an event loop.</p>
+
+ <h2>Performance Highlights</h2>
+
+ <p>pickleDB demonstrates strong performance for handling large-sized datasets:</p>
+
+ <table>
+ <thead>
+ <tr>
+ <th>Entries</th>
+ <th>Memory Load Time</th>
+ <th>Retrieval Time</th>
+ <th>Save Time</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><strong>1M</strong></td>
+ <td>0.68 sec</td>
+ <td>0.64 sec</td>
+ <td>0.03 sec</td>
+ </tr>
+ <tr>
+ <td><strong>10M</strong></td>
+ <td>7.48 sec</td>
+ <td>7.27 sec</td>
+ <td>0.22 sec</td>
+ </tr>
+ <tr>
+ <td><strong>50M</strong></td>
+ <td>43.36 sec</td>
+ <td>36.53 sec</td>
+ <td>1.09 sec</td>
+ </tr>
+ </tbody>
+ </table>
+
+ <p class="small-note">Tests were performed on a Dell XPS 9350 running Ubuntu 24.04 using pickleDB's async mode. <a href="https://gist.github.com/patx/025ed3a10482459f35c738228ebd0721">See the benchmark script used here.</a></p>
+
+ <h2>User Guide and Examples</h2>
+
+ <h3>Add or Update Data</h3>
+
+ <p>You can add or update key-value pairs using the <code>set()</code> method:</p>
+
+ <pre><code class="language-python"># Add a new key-value pair
+db.set('username', 'admin')
+
+# Or shorthand
+db['username'] = 'admin'
+
+# Update an existing key-value pair
+db.set('username', 'superadmin')
+print(db.get('username')) # Output: 'superadmin'
+</code></pre>
+
+ <p>Keys are automatically converted to strings, and values can be any <strong>JSON-serializable</strong> object.</p>
+
+ <h3>Retrieve Values</h3>
+
+ <p>You can retrieve a keys value using the <code>get()</code> method:</p>
+
+ <pre><code class="language-python"># Get the value for a key
+print(db.get('username')) # Output: 'superadmin'
+
+# Using Python syntax sugar
+db['username'] # Output: 'superadmin'
+
+# Attempt to retrieve a non-existent key
+print(db.get('nonexistent')) # Output: None
+</code></pre>
+
+ <h3>List All Keys</h3>
+
+ <p>You can get a list of all the keys currently in the database using the <code>all()</code> method:</p>
+
+ <pre><code class="language-python">db.set('item1', 'value1')
+db.set('item2', 'value2')
+
+print(db.all()) # Output: ['username', 'item1', 'item2']
+</code></pre>
+
+ <p><em>Note:</em> This method shows all keys currently loaded, it does <strong>not</strong> guarantee they are persisted to the disk (yet).</p>
+
+ <h3>Remove Keys</h3>
+
+ <p>To remove a key from the database use the <code>remove()</code> method:</p>
+
+ <pre><code class="language-python">db.remove('item1')
+print(db.all()) # Output: ['username', 'item2']
+</code></pre>
+
+ <h3>Purge the Database</h3>
+
+ <p>To remove all keys and their values from the database use the <code>purge()</code> method:</p>
+
+ <pre><code class="language-python">db.purge()
+print(db.all()) # Output: []
+</code></pre>
+
+ <h3>Saving Data</h3>
+
+ <p><strong>pickleDB does not auto-save by default</strong> for performance reasons. To persist data, call <code>save()</code> manually or use a context manager:</p>
+
+ <pre><code class="language-python">db.save() # Output: True
+
+# Context manager example
+with db:
+ db.set('foo', 'bar')
+ db.set('hello', 'world')
+# Automatically saves when exiting the context
+</code></pre>
+
+ <p><em>Note:</em> All the above methods work/display on the in-memory database. To persist any of the above methods actions you must call the <code>save()</code> method or use a context manager, as stated above.</p>
+
+ <h3>Asynchronous Usage</h3>
+
+ <p>pickleDB 1.4 uses a <strong>single unified class</strong> for both synchronous and asynchronous contexts.</p>
+
+ <pre><code class="language-python">import asyncio
+from pickledb import PickleDB
+
+async def main():
+ async with PickleDB('data.json') as db:
+ await db.set('score', 42)
+ print(await db.get('score')) # Output: 42
+
+asyncio.run(main())
+</code></pre>
+
+ <p>Just <code>await</code> any method when inside an async function/event-loop.</p>
+
+ <h3>Store and Retrieve Complex Data</h3>
+
+ <pre><code class="language-python"># Store a dictionary
+db.set('user', {'name': 'Alice', 'age': 30, 'city': 'Wonderland'})
+
+# Retrieve and modify
+user = db.get('user')
+user['age'] += 1
+db.set('user', user)
+
+print(db.get('user'))
+# Output: {'name': 'Alice', 'age': 31, 'city': 'Wonderland'}
+</code></pre>
+
+ <h3>Use Lists for Dynamic Data</h3>
+
+ <pre><code class="language-python">db.set('tasks', ['Write code', 'Test app', 'Deploy'])
+
+tasks = db.get('tasks')
+tasks.append('Celebrate')
+db.set('tasks', tasks)
+
+print(db.get('tasks'))
+# Output: ['Write code', 'Test app', 'Deploy', 'Celebrate']
+</code></pre>
+
+ <h3>Advanced Key Search</h3>
+
+ <p>You can filter keys dynamically using Python list comprehensions:</p>
+
+ <pre><code class="language-python">def get_keys_with_match(db_instance, match):
+ return [key for key in db_instance.all() if match in key]
+
+db.set('apple', 1)
+db.set('apricot', 2)
+db.set('banana', 3)
+
+print(get_keys_with_match(db, 'ap'))
+# Output: ['apple', 'apricot']
+</code></pre>
+
+ <h3>Namespaces</h3>
+
+ <p>Simulate namespaces using prefixes:</p>
+
+ <pre><code class="language-python">db.set('user:1', {'name': 'Alice'})
+db.set('user:2', {'name': 'Bob'})
+
+def get_namespace_keys(db_instance, namespace):
+ return [key for key in db_instance.all() if key.startswith(f"{namespace}:")]
+
+print(get_namespace_keys(db, 'user'))
+# Output: ['user:1', 'user:2']
+</code></pre>
+
+ <h3>Key Expiration (TTL)</h3>
+
+ <p>pickleDB doesn’t include TTL natively, but you can simulate it:</p>
+
+ <pre><code class="language-python">import time
+
+def set_with_expiry(db, key, value, ttl):
+ db.set(key, {'value': value, 'expires_at': time.time() + ttl})
+
+def get_if_not_expired(db, key):
+ data = db.get(key)
+ if data and time.time() < data['expires_at']:
+ return data['value']
+ db.remove(key)
+ return None
+
+set_with_expiry(db, 'session', 'active', ttl=5)
+time.sleep(3)
+print(get_if_not_expired(db, 'session')) # 'active'
+time.sleep(3)
+print(get_if_not_expired(db, 'session')) # None
+</code></pre>
+
+ <h3>Encrypted Storage</h3>
+
+ <pre><code class="language-python">from cryptography.fernet import Fernet
+
+key = Fernet.generate_key()
+cipher = Fernet(key)
+
+encrypted = cipher.encrypt(b"My secret data")
+db.set('secure', encrypted)
+
+decrypted = cipher.decrypt(db.get('secure'))
+print(decrypted.decode()) # Output: My secret data
+</code></pre>
+
+ <h3>Batch Operations</h3>
+
+ <pre><code class="language-python">def batch_set(db, items):
+ for key, value in items.items():
+ db.set(key, value)
+
+batch_set(db, {'k1': 'v1', 'k2': 'v2', 'k3': 'v3'})
+print(db.all())
+
+def batch_delete(db, keys):
+ for key in keys:
+ db.remove(key)
+
+batch_delete(db, ['k1', 'k2'])
+print(db.all())
+</code></pre>
+
+ <h3>Key Pattern Matching</h3>
+
+ <pre><code class="language-python">import re
+
+def get_keys_by_pattern(db, pattern):
+ regex = re.compile(pattern)
+ return [key for key in db.all() if regex.search(key)]
+
+db.set('user:1', {'name': 'Alice'})
+db.set('user:2', {'name': 'Bob'})
+db.set('admin:1', {'name': 'Charlie'})
+
+print(get_keys_by_pattern(db, r'user:\d'))
+# Output: ['user:1', 'user:2']
+</code></pre>
+
+ <h3>Signal Handling for Graceful Shutdowns</h3>
+
+ <pre><code class="language-python">import signal, sys
+from pickledb import PickleDB
+
+db = PickleDB('data.json')
+
+signal.signal(signal.SIGINT, lambda s, f: (db.save(), sys.exit(0)))
+signal.signal(signal.SIGTERM, lambda s, f: (db.save(), sys.exit(0)))
+
+db.set('key1', 'value1')
+print("Running... Press Ctrl+C to save and exit.")
+while True:
+ pass
+</code></pre>
+
+ <h3>Using pickleDB with Web Frameworks</h3>
+
+ <p>Example using <a href="https://patx.github.io/micropie">MicroPie</a>:</p>
+
+ <pre><code class="language-python">from uuid import uuid4
+from micropie import App
+from pickledb import PickleDB
+
+db = PickleDB('pastes.json')
+
+class Root(App):
+ async def index(self, paste_content=None):
+ if self.request.method == "POST":
+ pid = str(uuid4())
+ await db.set(pid, paste)
+ await db.save()
+ return self._redirect(f'/paste/{pid}')
+ return await self._render_template('index.html')
+
+ async def paste(self, paste_id):
+ paste = await db.get(paste_id)
+ return await self._render_template('paste.html', paste_id=paste_id, paste_content=paste)
+
+app = Root()
+</code></pre>
+
+ <h2>Core API Reference</h2>
+
+ <h3>Class: <code>PickleDB</code></h3>
+
+ <pre><code class="language-python">class PickleDB(location: str)
+</code></pre>
+
+ <p>A lightweight, JSON-backed key-value database. All data is kept in memory while loaded and written atomically to disk on <code>save()</code>.</p>
+
+ <h4>Parameters</h4>
+
+ <table>
+ <thead>
+ <tr>
+ <th>Name</th>
+ <th>Type</th>
+ <th>Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><code>location</code></td>
+ <td><code>str</code></td>
+ <td>Path to the JSON file backing the database. Tilde (<code>~</code>) is expanded.</td>
+ </tr>
+ </tbody>
+ </table>
+
+ <h3>Context Manager Support</h3>
+
+ <h4>Synchronous</h4>
+
+ <pre><code class="language-python">with PickleDB("data.json") as db:
+ db.set("foo", "bar")
+</code></pre>
+
+ <h4>Asynchronous</h4>
+
+ <pre><code class="language-python">async with PickleDB("data.json") as db:
+ await db.set("foo", "bar")
+</code></pre>
+
+ <p>On successful exit, the DB is automatically saved.</p>
+
+ <h3>Method Reference</h3>
+
+ <h4><code>load()</code></h4>
+
+ <pre><code class="language-python">load() -> None
+await load() -> None
+</code></pre>
+
+ <p>Loads the database into memory from disk if the file exists and contains valid JSON. Creates an empty database otherwise.</p>
+
+ <h4><code>save()</code></h4>
+
+ <pre><code class="language-python">save() -> bool
+await save() -> bool
+</code></pre>
+
+ <p>Writes the in-memory database to disk.</p>
+
+ <p>Returns <code>True</code> on success.</p>
+
+ <p>Notes:</p>
+ <ul>
+ <li>Uses a temporary file + <code>os.replace</code> for durability.</li>
+ <li>Automatically called on successful context-manager exit.</li>
+ </ul>
+
+ <h4><code>set()</code></h4>
+
+ <pre><code class="language-python">set(key: str, value: Any) -> bool
+await set(key: str, value: Any) -> bool
+</code></pre>
+
+ <p>Sets or updates a value for a key. <code>key</code> is coerced to <code>str</code>, and <code>value</code> must be JSON-serializable.</p>
+
+ <p>Returns <code>True</code>.</p>
+
+ <h5>Syntax Sugar</h5>
+
+ <pre><code class="language-python">db["username"] = "alice"
+</code></pre>
+
+ <h4><code>get()</code></h4>
+
+ <pre><code class="language-python">get(key: str, default: Any = None) -> Any
+await get(key: str, default: Any = None) -> Any
+</code></pre>
+
+ <p>Retrieves a value by key.</p>
+
+ <p>Returns:</p>
+ <ul>
+ <li>stored value</li>
+ <li><code>default</code> if key does not exist</li>
+ </ul>
+
+ <h5>Syntax Sugar</h5>
+
+ <pre><code class="language-python">value = db["username"]
+</code></pre>
+
+ <h4><code>remove()</code></h4>
+
+ <pre><code class="language-python">remove(key: str) -> bool
+await remove(key: str) -> bool
+</code></pre>
+
+ <p>Removes a key.</p>
+
+ <p>Returns:</p>
+ <ul>
+ <li><code>True</code> if removed</li>
+ <li><code>False</code> if not found</li>
+ </ul>
+
+ <h4><code>all()</code></h4>
+
+ <pre><code class="language-python">all() -> list[str]
+await all() -> list[str]
+</code></pre>
+
+ <p>Returns a list of all keys currently in memory.</p>
+
+ <h4><code>purge()</code></h4>
+
+ <pre><code class="language-python">purge() -> bool
+await purge() -> bool
+</code></pre>
+
+ <p>Clears the entire in-memory database.</p>
+
+ <p>Returns:</p>
+ <ul>
+ <li><code>True</code></li>
+ </ul>
+
</body>
</html>
diff --git a/pyproject.toml b/pyproject.toml
index 9ae005b..ed2451e 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -7,7 +7,7 @@ name = "pickledb"
version = "1.4"
description = "An ultra micro ASGI web framework"
keywords = ["pickle", "database", "json", "redis", "asyncio"]
-readme = "docs/README.md"
+readme = "README.md"
authors = [{ name = "Harrison Erd", email = "[email protected]" }]
license = {file = "LICENSE"}
dependencies = ["orjson>=3.11.5", "aiofiles>=25.1.0"]