first commit

This commit is contained in:
Longtail committed 2026-10-31 16:15:04 +08:00
commit 1e6504b4a2
41 files changed
+4964

No files matched your search

+10
View File
@@ -0,0 +1,10 @@
data/
downloads/
local-records/
test/history/
test/artifacts/
发布去重记录.json
__pycache__/
*.py[cod]
.venv/
.DS_Store
+97
View File
@@ -0,0 +1,97 @@
# 新闻工作台
将新闻下载、文章打包、浏览器导入、排版与换号联动整合到本地控制台。
**当前版本:0.1。可完成下载 → 入队 → 导入 → 排版 → 等待人工发表。结果页自动核对按用户决定暂留待接入;人工提交后会记录候选结果并暂停,不会自动计数、换号或处理下一篇。** 换号调度和计数接口已完成离线验证,完整在线链路仍需补齐结果页 HTML 后验证。
## 启动
双击本目录的 `start.command`,或在本目录运行:
```sh
uv run --no-project --python 3.12 --with-requirements requirements.txt python app.py
```
默认打开 http://127.0.0.1:8770/ 。保留启动窗口,按 Ctrl+C 停止本地服务。修改端口使用 `--port 8772`;不自动打开浏览器使用 `--no-browser`。首次运行 uv 可能下载 Python 或依赖,后续可以复用缓存。
页面必须通过本地服务打开,不要直接双击 `static/index.html`。Python 依赖仅用于下载和内容处理;浏览器脚本没有远程脚本依赖。
## 浏览器安装与配对
1. 在油猴安装控制台“浏览器连接”区域的两个脚本:
- `browser/local-bridge.user.js`:本地连接、配对与常驻终止按钮。
- `browser/auto-news.user.js`:导入、排版和任务执行。它由 `importer.js`、`formatter.js`、`runner.js` 构建。
2. 停用旧的独立导入与排版油猴脚本,避免同一编辑器重复挂载。
3. 换号插件仍由用户安装,位于项目根目录的 `dww11-account-switch-browser-extension/`,没有复制到本目录。本次在其 `content.js` 中增加了不含凭据的状态属性及终止事件监听。请在浏览器扩展管理页重新加载**这个原安装目录**,再刷新网站。旧 ZIP 未重新打包,不包含本次修改。
4. 在控制台点击“显示本次连接码”。在论坛左下角“新闻工作台 · 本地连接”展开设置,填写端口和连接码并保存。
5. 服务重启后连接码会改变,需要重新配对;任务和计数保留,但服务不会自动恢复运行。
本机 HTTP 通信通过油猴特权请求完成,不要求论坛跨域开放本地服务。真实浏览器的脚本管理器配置、本地网络权限和网站编辑器兼容性仍需首次实机验证。配对码保存在油猴存储,不写入页面全局变量。下载 API Key 不进入文章包,不发给论坛。
## 操作流程
1. 填写 Key,获取频道、选择数量并下载。完整保存的文章会自动校验并进入队列;下载失败和入队失败分别显示,已保存的文章不因入队失败而删除。
2. 也可展开“导入已有文章目录”,填写单篇目录或含多个文章子目录的根目录,点击检查。留空表示扫描本工具的 `downloads/`。扫描只处理根目录的一层文章子目录;文件须已完整落盘。
3. 填写目标版块 `fid`、每账号最少/最多篇数和处理间隔。先核对历史发帖记录,再勾选确认并保存。版块和篇数不会替用户选择。
4. 登录论坛,建议只保留一个执行标签。点击控制台“开始 / 继续”。执行器识别账号后领取一篇任务,进入指定新帖页、导入、排版,停止在人工审核阶段。
5. 在论坛检查账号、版块、标题、图片位置、正文和来源,并亲手点击“发表帖子”。程序不调用发帖提交函数,不发送发帖请求,也不自动保存草稿。
6. 当前版本会记录人工点击和提交快照,在跳转后的候选页面或等待超时后标记“发布结果待核对”并暂停。**需要补充结果页 HTML 后,才能自动核对受理结果并接续下一篇。**
待补齐结果页自动核对后:审核中算网站已受理一次,但不算已确认公开;达到账号随机目标篇数且仍有待处理文章时,回首页调用插件换号,核对新 UID 与插件空闲状态后创建下一账号批次。没有待处理文章时直接结束,不为达到数量重发。
## 终止、恢复与冲突
- 网站左下角常驻“终止流程”,点击后立即取消本标签可取消的导入,并通知其他安装脚本的标签和换号插件;持久状态禁止继续领取任务。
- 控制台“终止流程”还会停止下载,浏览器在下一次连接检查时停止(正常轮询约 2 秒)。网络断开时无法承诺固定延迟;网站按钮可立即停止本标签。已到达网站的图片、登录或发帖请求无法撤回。
- 暂停/终止后必须明确点击“开始 / 继续”。下载停止等待当前网络请求/单篇保存结束,不伪装成已经撤回在途请求。
- 非空编辑器、人工修改、账号变化、缺图、未知来源或上传失败会停在“需要处理”,不弹自动确认覆盖旧稿。
- 导入中刷新后不会重新上传。先核对网站附件和草稿,再决定如何处理。若需要换标签,先在控制台暂停,再在目标标签点击“接管已暂停任务”。接管不会自动重新导入内容。
- 只有人工确认网站**未受理**且已清理编辑器/草稿后,才能通过队列“人工核对 / 处理”重新排队或移出。审核中、已受理和结果不明不应重试。
- 换号插件暂停后需要通过插件现有“继续未完成流程”恢复,不重复抽号。若账号未发生变化,调度器不会猜测换号成功。
## 文件与数据
```text
app.py / store.py / articles.py 本地接口、任务队列、文章校验
modules/downloader/ 下载器副本与成功保存回调
modules/package.py 文章包生成器副本
browser/ 页面执行器、排版和连接脚本
static/ 操作前端
start.command 启动入口
build_browser.py 重建合并油猴脚本
test/ 离线测试、模拟素材、历史测试截图
data/queue.sqlite3 运行任务、计数、受理去重记录(本地)
downloads/ 本工具下载结果(本地)
```
- `test/history/` 保存从本目录移入的两张演示截图与演示任务 JSON。根目录其他测试文件未搬动。
- `发布去重记录.json` 是实际受理证据,继续留在原处,并在服务启动时迁入去重库。它不代表完整历史已对账。原文件可找到时还建立内容指纹;原文件丢失时仍保留原 Markdown 摘要。
- 原始下载文件不改写。长标题生成保守 UTF-16 长度不超过 80 的派生标题并加 `...`,原题仍展示在队列;网站真实计数规则尚需实测。
- 排版仅处理与文章来源一致的末尾标准尾注,保留正文中的其他“来源”文字。图片顺序、标识改变会回滚排版。
- WebP/AVIF/SVG 当前明确拦截,需用户另存支持格式并更新 Markdown;没有静默丢图或自动转码。单图须小于 20 MiB,图片 base64 总大小不超过 48 MiB。
- 本地库含文章内容快照和账号标识,不含账号密码、Cookie 或上传凭据。运行数据和历史截图已加入忽略规则。
## 验证
不连接新闻 API 或真实论坛:
```sh
uv run --no-project --python 3.12 --with-requirements requirements.txt python -m unittest discover -s test -p 'test_*.py' -v
node --test test/importer.test.cjs test/bridge.test.cjs
python3 build_browser.py
```
浏览器回归需要 Playwright 和本机 Chrome,使用临时独立浏览器,不使用用户配置或账号:
```sh
# 终端一:临时模拟数据服务,端口 8771
uv run --no-project --python 3.12 --with-requirements requirements.txt python test/serve_fixture.py
# 终端二:Node 环境需能加载 playwright;CHROME_PATH 可指定浏览器可执行文件
node test/browser.test.cjs
```
浏览器回归拦截所有真实论坛请求,仅使用模拟 HTML。截图输出到 `test/artifacts/`。本轮离线通过不代表已在用户登录态的真实网站上完成上传、换号或发布。
## 下一步需要的页面资料
按用户决定,等待结果页 HTML:帖子标题、首帖作者及 UID 链接、版块及 fid 链接、“审核中”提示、公开帖子与失败提示。不要提供 Cookie、密码或表单令牌。将据此完成 `runner.js` 的结果观察适配,调用已经实现的受理记录接口,联通后续计数和换号。
+205
View File
@@ -0,0 +1,205 @@
#!/usr/bin/env python3
"""Local controller: serves UI and authenticated browser bridge on loopback only."""
import argparse
import hmac
import importlib.util
import json
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
import secrets
import sys
import threading
from urllib.parse import urlsplit
import webbrowser
ROOT = Path(__file__).resolve().parent
sys.path.insert(0, str(ROOT / 'modules/downloader'))
spec = importlib.util.spec_from_file_location('news_downloader', ROOT / 'modules/downloader/app.py')
downloader = importlib.util.module_from_spec(spec)
spec.loader.exec_module(downloader)
from store import Store
class Controller:
def __init__(self, data=ROOT / 'data', downloads=ROOT / 'downloads', ledger=ROOT / '发布去重记录.json'):
self.store = Store(Path(data) / 'queue.sqlite3', ledger)
self.ui_token, self.bridge_token = secrets.token_urlsafe(32), secrets.token_urlsafe(32)
self.manager = downloader.Downloader(downloads, on_saved=self.saved)
self.scan_lock = threading.Lock()
def saved(self, article, result, keys):
directory = Path(result.directory)
marker = directory / '.article.json'
pending = directory / '.article.json.tmp'
pending.write_text(json.dumps({'fingerprints': keys, 'source_url': result.source_url}, ensure_ascii=False), encoding='utf-8')
pending.replace(marker)
self.store.enqueue(directory)
def scan(self, root):
if self.manager.thread and self.manager.thread.is_alive():
raise ValueError('下载进行中,请等待完成后扫描;新下载文章会自动入队')
root = Path(root).expanduser().resolve(strict=True)
if not root.is_dir():
raise ValueError('文章根目录不存在')
if not self.scan_lock.acquire(blocking=False):
raise ValueError('正在扫描,请稍候')
try:
directories = [root] if list(root.glob('*.md')) else sorted(p for p in root.iterdir() if p.is_dir() and not p.name.startswith('.'))
result = []
for directory in directories:
try:
status = self.store.enqueue(directory)
except (ValueError, OSError) as error:
status = str(error)
result.append({'directory': directory.name, 'status': status})
return {'results': result}
finally:
self.scan_lock.release()
def ui(self, route, body):
if route == 'state':
return {**self.store.state(), 'download': self.manager.snapshot()}
if route == 'pair':
return {'token': self.bridge_token}
if route == 'channels':
key = str(body.get('key', '')).strip()
if not key or len(key) > 512:
raise ValueError('请填写 API Key')
items = downloader.result_list(downloader.api_request(key, 'channel'))
if not items or not all(isinstance(x, str) and x for x in items):
raise ValueError('频道响应格式不正确')
return {'channels': items}
if route == 'download':
key, channel, target = str(body.get('key', '')).strip(), body.get('channel'), body.get('target')
if not key or len(key) > 512 or not isinstance(channel, str) or not channel.strip() or type(target) is not int or not 1 <= target <= 100000:
raise ValueError('请填写 Key、频道和有效下载数量')
self.manager.start(key, channel, target)
return {'ok': True}
if route == 'download-stop':
self.manager.stop.set()
return {'ok': True}
if route == 'scan':
return self.scan(body.get('root', '')) if body.get('root') else self.scan(self.manager.root)
if route == 'configure':
self.store.configure(body)
return {'ok': True}
if route in ('start', 'pause', 'stop'):
if route == 'stop':
self.manager.stop.set()
return self.store.control(route)
if route == 'resolve':
return self.store.resolve(body)
if route == 'history':
records = body.get('records')
if not isinstance(records, list):
raise ValueError('历史记录需包含 records 数组')
self.store.import_history(records)
return {'ok': True}
raise ValueError('未知操作')
def bridge(self, route, body):
methods = {'heartbeat': self.store.heartbeat, 'next': self.store.next, 'package': self.store.package,
'event': self.store.event, 'accept': self.store.accept, 'switch-begin': self.store.switch_begin,
'takeover': self.store.takeover, 'current': self.store.current}
if route == 'stop':
self.manager.stop.set()
return self.store.control('stop')
if route not in methods:
raise ValueError('未知浏览器操作')
return methods[route](body)
def make_handler(controller):
class Handler(BaseHTTPRequestHandler):
def log_message(self, *args):
pass # Never echo request bodies, API keys or pairing tokens.
def send(self, code, payload, mime='application/json; charset=utf-8'):
content = payload if isinstance(payload, bytes) else json.dumps(payload, ensure_ascii=False).encode()
self.send_response(code)
self.send_header('Content-Type', mime)
self.send_header('Content-Length', str(len(content)))
self.send_header('Cache-Control', 'no-store')
self.send_header('X-Content-Type-Options', 'nosniff')
self.send_header('Content-Security-Policy', "default-src 'self'; script-src 'self'; style-src 'self'; connect-src 'self'; frame-ancestors 'none'; base-uri 'none'")
self.end_headers()
try:
self.wfile.write(content)
except (BrokenPipeError, ConnectionResetError):
pass
def local(self):
return self.headers.get('Host') in (f'127.0.0.1:{self.server.server_port}', f'localhost:{self.server.server_port}')
def origin_ok(self):
origin = self.headers.get('Origin')
return not origin or origin in (f'http://127.0.0.1:{self.server.server_port}', f'http://localhost:{self.server.server_port}')
def do_GET(self):
if not self.local() or not self.origin_ok():
return self.send(403, {'error': '仅允许本机控制台访问'})
route = urlsplit(self.path).path
if route == '/api/session':
return self.send(200, {'token': controller.ui_token})
files = {'/': ('static/index.html', 'text/html; charset=utf-8'),
'/app.js': ('static/app.js', 'text/javascript; charset=utf-8'),
'/style.css': ('static/style.css', 'text/css; charset=utf-8'),
'/browser/auto-news.user.js': ('browser/auto-news.user.js', 'text/javascript; charset=utf-8'),
'/browser/local-bridge.user.js': ('browser/local-bridge.user.js', 'text/javascript; charset=utf-8')}
if route not in files:
return self.send(404, {'error': '不存在'})
name, mime = files[route]
file = ROOT / name
if not file.exists():
return self.send(404, {'error': '请先运行 build_browser.py'})
return self.send(200, file.read_bytes(), mime)
def do_POST(self):
if not self.local():
return self.send(403, {'error': 'Host 不匹配'})
route = urlsplit(self.path).path
bridge = route.startswith('/bridge/')
expected = controller.bridge_token if bridge else controller.ui_token
provided = self.headers.get('X-Session-Token', '')
if not hmac.compare_digest(expected, provided) or (not bridge and not self.origin_ok()):
return self.send(403, {'error': '连接凭据无效,请重新配对'})
if not bridge and not route.startswith('/api/'):
return self.send(404, {'error': '不存在'})
try:
length = int(self.headers.get('Content-Length', 0))
if not 0 < length <= 2 * 1024 * 1024:
raise ValueError('请求大小无效')
body = json.loads(self.rfile.read(length))
if not isinstance(body, dict):
raise ValueError('请求必须是对象')
result = (controller.bridge if bridge else controller.ui)(route.rsplit('/', 1)[1], body)
self.send(200, result)
except (ValueError, KeyError, OSError) as error:
self.send(400, {'error': str(error)[:1000]})
except Exception:
self.send(500, {'error': '内部错误;请停止当前任务并检查本地服务'})
return Handler
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--port', type=int, default=8770)
parser.add_argument('--no-browser', action='store_true')
args = parser.parse_args()
controller = Controller()
server = ThreadingHTTPServer(('127.0.0.1', args.port), make_handler(controller))
url = f'http://127.0.0.1:{args.port}/'
print('新闻工作台:' + url, flush=True)
if not args.no_browser:
webbrowser.open(url)
try:
server.serve_forever()
except KeyboardInterrupt:
controller.manager.stop.set()
controller.store.control('stop')
finally:
server.server_close()
if __name__ == '__main__':
main()
+116
View File
@@ -0,0 +1,116 @@
"""Local article validation and portable packages. Never changes downloaded files."""
import hashlib
import json
from pathlib import Path
import re
import unicodedata
from urllib.parse import unquote
from modules.package import make_package
MAX_PACKAGE_BYTES = 48 * 1024 * 1024
FOOTER = re.compile(r'(?:^|\n)来源:([^\n]+),如有侵权,请联系删除。\s*$')
def validate_images(body, directory):
"""Conservative inline-image preflight; the importer remains the final parser."""
if re.search(r'!\[[^\]]*\]\s*\[', body):
raise ValueError('不支持引用式图片,请改成内联图片路径')
for start in re.finditer(r'!\[[^\]]*\]\(', body):
pos, depth, quote, angle = start.end(), 1, '', False
begin = pos
while pos < len(body):
ch = body[pos]
if ch == '\\':
pos += 2
continue
if quote:
if ch == quote:
quote = ''
elif not angle and depth == 1 and body[pos - 1].isspace() and ch in ('"', "'"):
quote = ch
elif ch == '<':
angle = True
elif ch == '>':
angle = False
elif not angle and ch == '(':
depth += 1
elif not angle and ch == ')':
depth -= 1
if not depth:
break
pos += 1
if depth:
raise ValueError('图片引用括号不完整')
target = body[begin:pos].strip()
target = target[1:target.index('>')] if target.startswith('<') and '>' in target else re.sub(r'\s+["\'][\s\S]*["\']\s*$', '', target)
target = unquote(re.sub(r'\\([()\\ ])', r'\1', target))
if re.match(r'^(?:[a-zA-Z][\w+.-]*:|/|\\)', target):
raise ValueError('图片必须使用目录内相对路径:' + target)
image = (directory / target).resolve()
if not image.is_relative_to(directory) or not image.is_file():
raise ValueError('缺图或图片路径超出目录:' + target)
if image.suffix.lower() not in ('.jpg', '.jpeg', '.png', '.gif'):
raise ValueError('不支持的配图格式:' + target)
if not 0 < image.stat().st_size < 20 * 1024 * 1024:
raise ValueError('配图为空或达到 20 MiB:' + target)
def short_title(title, budget=80):
# Conservative UTF-16 budget: do not split astral characters / surrogate pairs.
if len(title.encode('utf-16-le')) // 2 <= budget:
return title
out = ''
for ch in title:
if len((out + ch).encode('utf-16-le')) // 2 > budget - 3:
break
out += ch
return out + '...'
def inspect_article(directory):
directory = Path(directory).resolve(strict=True)
candidates = sorted(directory.glob('*.md'))
if len(candidates) != 1:
raise ValueError('每个文章目录必须恰好包含一个 Markdown 文件')
md = candidates[0]
if not md.resolve().is_relative_to(directory):
raise ValueError('Markdown 不得链接到目录外')
raw = md.read_text(encoding='utf-8-sig')
heading = re.match(r'^\s*#\s+([^\n]+)\n?', raw)
title = (heading[1] if heading else md.stem).strip()
body = raw[heading.end():].strip() if heading else raw.strip()
footer = FOOTER.search(body)
if not footer:
raise ValueError('文末缺少标准来源尾注,请补充来源后再扫描')
source = footer[1].strip()
body = body[:footer.start()].strip()
if not title or not body:
raise ValueError('标题或正文为空')
validate_images(body, directory)
for file in directory.rglob('*'):
if file.suffix.lower() in ('.webp', '.avif', '.svg'):
raise ValueError('发现暂不支持的图片格式:' + file.name + ';请另存为 JPG/PNG/GIF 并更新引用')
pack = make_package(directory)
if sum(len(i['base64']) for i in pack['images']) > MAX_PACKAGE_BYTES:
raise ValueError('文章包超过 48 MiB,请减少单篇图片体积')
pack['title'] = short_title(title)
# Preserve body and exact final footer for the formatter; the original file stays intact.
pack['markdown'] = '# ' + pack['title'] + '\n\n' + body + '\n\n来源:' + source + ',如有侵权,请联系删除。\n'
content = unicodedata.normalize('NFC', body)
# Hash file contents, not directory/title; renamed copies share their article identity.
image_hashes = []
for item in pack['images']:
import base64
digest = hashlib.sha256(base64.b64decode(item['base64'])).hexdigest()
content = content.replace(item['path'], 'image:' + digest)
image_hashes.append(digest)
identity = hashlib.sha256((content + '\n' + '\n'.join(sorted(set(image_hashes)))).encode()).hexdigest()
metadata = {}
marker = directory / '.article.json'
if marker.is_file():
metadata = json.loads(marker.read_text())
return dict(article_key=identity, markdown_sha256=hashlib.sha256(md.read_bytes()).hexdigest(),
directory=str(directory), original_title=title, title=pack['title'], source=source,
image_count=len(pack['images']), package=pack,
source_keys=metadata.get('fingerprints', []))
+850
View File
@@ -0,0 +1,850 @@
// ==UserScript==
// @name 新闻工作台 · 导入排版与流程
// @namespace local.dww11.auto-news
// @version 0.1.0
// @match https://www.dww11.com/*
// @match https://dww11.com/*
// @match https://house.dww11.com/*
// @grant none
// @run-at document-idle
// @noframes
// ==/UserScript==
(function (root, factory) {
'use strict';
const core = factory();
if (typeof module === 'object' && module.exports) module.exports = core;
else core.mount(root);
})(typeof window === 'undefined' ? null : window, function () {
'use strict';
const VERSION = '1.0.0';
const MAX_IMAGE_SIZE = 20 * 1024 * 1024;
const IMAGE_EXT = /\.(?:jpe?g|png|gif)$/i;
function normalizePath(path) {
const parts = [];
const value = String(path).replace(/\\/g, '/').normalize('NFC');
if (/^(?:\/|[a-z]:|[a-z]+:)/i.test(value)) throw new Error('请使用目录内的相对路径:' + path);
for (const part of value.split('/')) {
if (!part || part === '.') continue;
if (part === '..') {
if (!parts.length) throw new Error('图片路径超出所选目录:' + path);
parts.pop();
} else parts.push(part);
}
return parts.join('/');
}
function imagePath(mdPath, target) {
if (/^(?:[a-z][\w+.-]*:|\/\/|\/)/i.test(target)) throw new Error('图片必须在所选目录内,请先下载:' + target);
let decoded;
try { decoded = decodeURIComponent(target); } catch { throw new Error('图片路径编码无效:' + target); }
return normalizePath(mdPath.slice(0, mdPath.lastIndexOf('/') + 1) + decoded);
}
// A small, intentionally bounded Markdown reader. Raw HTML is never executed.
// Links/images support balanced parentheses, angle destinations and optional titles.
function readLink(text, start) {
const image = text.startsWith('![', start);
if (!image && text[start] !== '[') return null;
const left = start + (image ? 2 : 1);
let end = left;
for (; end < text.length; end++) {
if (text[end] === '\\') { end++; continue; }
if (text[end] === ']') break;
}
if (text.slice(end, end + 2) !== '](') return null;
let pos = end + 2, depth = 1, quoted = '', angle = false;
for (; pos < text.length; pos++) {
const ch = text[pos];
if (ch === '\\') { pos++; continue; }
if (quoted) { if (ch === quoted) quoted = ''; continue; }
if (!angle && depth === 1 && /\s/.test(text[pos - 1]) && (ch === '"' || ch === "'")) { quoted = ch; continue; }
if (ch === '<') { angle = true; continue; }
if (ch === '>') { angle = false; continue; }
if (!angle && ch === '(') depth++;
if (!angle && ch === ')' && --depth === 0) break;
}
if (depth) return null;
let target = text.slice(end + 2, pos).trim();
const destination = target.match(/^<([^>]+)>\s*(?:["'][\s\S]*["'])?$/);
target = destination ? destination[1] : target.replace(/\s+["'][\s\S]*["']\s*$/, '');
target = target.replace(/\\([()\\ ])/g, '$1');
return { image, label: text.slice(left, end), target, end: pos + 1 };
}
function inline(text, onImage) {
let out = '';
for (let pos = 0; pos < text.length;) {
if (text[pos] === '\\' && pos + 1 < text.length) { out += text[pos + 1]; pos += 2; continue; }
const link = readLink(text, pos);
if (link) {
if (link.image) out += onImage(link.target, link.label);
else if (/^(https?:\/\/|mailto:)/i.test(link.target) && !/[\[\]\r\n"']/.test(link.target)) {
out += '[url=' + link.target + ']' + inline(link.label, onImage) + '[/url]';
} else out += link.label + ' (' + link.target + ')';
pos = link.end;
continue;
}
if (text[pos] === '`') {
const end = text.indexOf('`', pos + 1);
if (end > pos + 1) { out += text.slice(pos + 1, end); pos = end + 1; continue; }
}
let matched = false;
for (const [mark, tag] of [['**', 'b'], ['__', 'b'], ['~~', 's'], ['*', 'i'], ['_', 'i']]) {
if (!text.startsWith(mark, pos) || (mark === '_' && /\w/.test(text[pos - 1] || ''))) continue;
const end = text.indexOf(mark, pos + mark.length);
if (end <= pos + mark.length) continue;
out += '[' + tag + ']' + inline(text.slice(pos + mark.length, end), onImage) + '[/' + tag + ']';
pos = end + mark.length; matched = true; break;
}
if (!matched) out += text[pos++];
}
return out;
}
function markdownToBBCode(markdown, onImage) {
const lines = markdown.replace(/\r\n?/g, '\n').split('\n');
const output = [];
let list = '', quote = false, code = false, fence = '';
const closeList = () => { if (list) output.push('[/list]'); list = ''; };
for (let index = 0; index < lines.length; index++) {
const line = lines[index];
const codeMark = line.match(/^\s*(`{3,}|~{3,})/);
if (codeMark && (!code || codeMark[1][0] === fence)) {
closeList();
output.push(code ? '[/code]' : '[code]');
code = !code; fence = codeMark[1][0]; continue;
}
if (code) { output.push(line.replace(/\[\/code\]/gi, '[/code]')); continue; }
const quoted = /^\s*>\s?/.test(line);
if (quoted !== quote) { closeList(); output.push(quoted ? '[quote]' : '[/quote]'); quote = quoted; }
const content = quoted ? line.replace(/^\s*>\s?/, '') : line;
const item = content.match(/^\s*(?:([-+*])|\d+[.)])\s+(.+)$/);
if (item) {
const type = item[1] ? '[list]' : '[list=1]';
if (list !== type) { closeList(); output.push(type); list = type; }
output.push('[*]' + inline(item[2], onImage)); continue;
}
closeList();
const heading = content.match(/^\s{0,3}(#{1,6})\s+(.+?)\s*#*$/);
if (heading) output.push('[b]' + inline(heading[2], onImage) + '[/b]');
else if (/^\s*(?:---+|\*\*\*+|___+)\s*$/.test(content)) output.push('[hr]');
else output.push(inline(content, onImage));
}
closeList();
if (quote) output.push('[/quote]');
if (code) output.push('[/code]');
return output.join('\n').trim();
}
function compile(markdown, mdPath, fileMap, titleOverride) {
let body = String(markdown).replace(/^\uFEFF/, '').replace(/\r\n?/g, '\n');
const firstHeading = body.match(/^\s*#\s+([^\n]+)\n?/);
const title = String(titleOverride || firstHeading?.[1] || mdPath.split('/').pop().replace(/\.md$/i, '')).trim();
if (firstHeading) body = body.slice(firstHeading[0].length).trim();
if (!title || Array.from(title).length > 80) throw new Error('标题为空或超过 80 字,请修改 Markdown 标题后重试。');
if (!body.trim()) throw new Error('正文为空。');
const warnings = [];
if (/<\/?[a-z][^>]*>/i.test(body)) warnings.push('存在 HTML,将作为文字保留;请检查排版。');
if (/^\s*\|.*\|\s*$/m.test(body)) warnings.push('表格暂按原文保留,需手动检查。');
if (/!\[[^\]]*\]\s*\[[^\]]*\]/.test(body) || /^\s*\[[^\]]+\]:/m.test(body)) {
throw new Error('暂不支持引用式 Markdown 图片/链接,请改成 ![](图片路径) 或 [文字](网址)。');
}
const images = new Map();
// Validate every image before any network request; repeated references upload once.
markdownToBBCode(body, (target) => {
const path = imagePath(mdPath, target);
const file = fileMap.get(path);
if (!file) throw new Error('找不到配图:' + path);
if (!IMAGE_EXT.test(path)) throw new Error('网站仅支持 JPG、JPEG、PNG、GIF:' + path);
if (!file.size || file.size >= MAX_IMAGE_SIZE) throw new Error('配图为空或达到 20 MB:' + path);
images.set(path, file);
return '[图片:' + path + ']';
});
return { title, body, mdPath, images, warnings,
render(aids) {
return markdownToBBCode(body, target => {
const path = imagePath(mdPath, target);
const aid = aids.get(path);
if (!/^[1-9]\d*$/.test(String(aid))) throw new Error('配图尚未上传:' + path);
return '[attachimg]' + aid + '[/attachimg]';
});
}
};
}
async function prepareFiles(files, markdownPath) {
const fileMap = new Map();
for (const file of Array.from(files)) {
const path = normalizePath(file.webkitRelativePath || file.name);
if (fileMap.has(path)) throw new Error('文件路径重复:' + path);
fileMap.set(path, file);
}
const candidates = Array.from(fileMap.keys()).filter(p => /\.md$/i.test(p)).sort();
if (!candidates.length) throw new Error('所选目录中没有 .md 文件。');
if (!markdownPath && candidates.length > 1) return { candidates, files: Array.from(files) };
const mdPath = markdownPath || candidates[0];
const md = fileMap.get(mdPath);
if (!md) throw new Error('指定的 Markdown 文件不存在。');
return compile(await md.text(), mdPath, fileMap);
}
async function preparePackage(data) {
if (!data || data.schemaVersion !== 1 || typeof data.markdown !== 'string' || !Array.isArray(data.images)) {
throw new Error('文章包格式错误,需要 schemaVersion: 1、markdown 和 images。');
}
const map = new Map();
for (const item of data.images) {
const path = normalizePath(item.path);
if (map.has(path)) throw new Error('文章包图片路径重复:' + path);
let blob = item.blob;
if (!blob && typeof item.base64 === 'string') {
const binary = atob(item.base64);
blob = new Blob([Uint8Array.from(binary, ch => ch.charCodeAt(0))], { type: item.mime || 'application/octet-stream' });
}
if (!blob || typeof blob.arrayBuffer !== 'function') throw new Error('文章包缺少图片数据:' + path);
map.set(path, blob);
}
return compile(data.markdown, normalizePath(data.markdownPath || 'article.md'), map, data.title);
}
function parseAid(text) {
const value = String(text).trim();
// The site's actual WebUploader adapter uses a positive integer response.
if (!/^(?:[1-9]\d*|"[1-9]\d*")$/.test(value)) throw new Error('服务器未返回有效附件编号(可能登录过期、超出权限或上传失败)。');
return value.replace(/"/g, '');
}
function mount(win) {
const doc = win.document;
const params = new URLSearchParams(win.location.search);
if (params.get('mod') !== 'post' || params.get('action') !== 'newthread' || !doc.querySelector('#subject') || !doc.querySelector('#e_textarea')) return;
if (win.DWW11Importer) return;
let article = null, busy = false, candidates = null, lastResult = null, savedBackup = null, stageRevision = 0;
const cache = new Map(), urls = [], logs = [];
const host = doc.createElement('div');
host.id = 'dww11-importer';
host.style.cssText = 'position:fixed;right:18px;bottom:18px;z-index:2147483000';
const shadow = host.attachShadow({ mode: 'open' });
shadow.innerHTML = `<style>
:host{all:initial}*{box-sizing:border-box}details{font:14px/1.6 system-ui,sans-serif;color:#203044;background:#fff;border:1px solid #d4dee8;border-radius:12px;box-shadow:0 6px 28px #0003;width:360px;max-width:92vw}summary{cursor:pointer;padding:12px 16px;font-weight:650}section{padding:0 16px 16px;max-height:70vh;overflow:auto}button,select{font:inherit;padding:7px 10px;border-radius:6px;border:1px solid #b8c9da;background:#f4f8fc;color:#203044;cursor:pointer;margin:4px 4px 4px 0}button.primary{background:#1767b3;color:white;border-color:#1767b3}button:disabled{opacity:.5;cursor:default}p{margin:8px 0}small{color:#617183}pre{font:12px/1.6 system-ui;white-space:pre-wrap;overflow-wrap:anywhere;max-height:170px;overflow:auto;background:#f4f6f9;padding:8px}img{max-width:90px;max-height:80px;margin:4px;object-fit:contain}.error{color:#b32424}.success{color:#18713e}select{max-width:100%}[hidden]{display:none!important}
</style><details open><summary>本地文章导入 · 1.0</summary><section>
<small>选择整个文章文件夹,检查后上传配图并填写。发布由你操作。网站自带的自动保存仍可能运行。</small>
<p><button id="choose" type="button">选择文章目录</button><button id="package" type="button">导入文章包</button></p>
<input id="folder" type="file" webkitdirectory multiple hidden><input id="json" type="file" accept=".json,application/json" hidden>
<select id="md" hidden aria-label="选择 Markdown"></select>
<p id="status" role="status" aria-live="polite">尚未选择目录</p><div id="preview" hidden><strong id="title"></strong><pre id="body"></pre><div id="images"></div><p id="warnings"></p></div>
<button id="import" class="primary" type="button" disabled>上传配图并填写</button>
<button id="diagnostics" type="button">导出诊断</button><pre id="log" hidden></pre>
</section></details>`;
doc.body.appendChild(host);
const $ = id => shadow.getElementById(id);
const summary = () => ({ version: VERSION, busy, ready: !!article, title: article?.title || null,
markdownPath: article?.mdPath || null, imageCount: article?.images.size || 0,
warnings: article?.warnings || [], result: lastResult });
function notify(phase, message, extra = {}) {
$('status').textContent = message;
$('status').className = phase === 'error' ? 'error' : phase === 'complete' ? 'success' : '';
logs.push({ time: new Date().toISOString(), phase, message, ...extra });
$('log').hidden = false;
$('log').textContent = logs.slice(-8).map(l => l.message).join('\n');
win.dispatchEvent(new CustomEvent('dww11-importer:progress', { detail: { phase, message, ...extra } }));
}
function releasePreviews() { while (urls.length) URL.revokeObjectURL(urls.pop()); }
function clearStage() {
article = null; lastResult = null; candidates = null; releasePreviews();
$('preview').hidden = true; $('md').hidden = true; $('import').disabled = true;
}
function showArticle(prepared) {
article = prepared; lastResult = null; releasePreviews();
$('preview').hidden = false;
$('title').textContent = article.title;
$('body').textContent = article.body;
$('warnings').textContent = article.warnings.join(';');
$('images').replaceChildren();
for (const [path, blob] of article.images) {
const img = doc.createElement('img');
img.src = URL.createObjectURL(blob); urls.push(img.src); img.alt = path; img.title = path;
$('images').appendChild(img);
}
$('import').disabled = false;
notify('ready', `已检查:${article.images.size} 张图片。点击“上传配图并填写”开始导入。`);
return summary();
}
async function stageFiles(files, options = {}) {
if (busy) throw new Error('正在导入,请稍候。');
clearStage();
const revision = ++stageRevision;
const list = Array.from(files);
const prepared = await prepareFiles(list, options.markdownPath);
if (revision !== stageRevision) return { cancelled: true };
if (prepared.candidates) {
candidates = prepared.files;
$('md').replaceChildren(new Option('请选择目录中的 Markdown 文件', ''));
for (const path of prepared.candidates) $('md').appendChild(new Option(path, path));
$('md').hidden = false;
notify('choose-markdown', '目录中有多个 Markdown 文件,请选择一篇。');
return { candidates: prepared.candidates };
}
return showArticle(prepared);
}
async function stagePackage(data) {
if (busy) throw new Error('正在导入,请稍候。');
clearStage();
const revision = ++stageRevision;
const prepared = await preparePackage(data);
if (revision !== stageRevision) return { cancelled: true };
return showArticle(prepared);
}
const snapshot = () => ({ title: doc.querySelector('#subject').value,
mode: Number(win.wysiwyg), text: win.getEditorContents() });
function assertEditor() {
for (const name of ['getEditorContents', 'switchEditor']) {
if (typeof win[name] !== 'function') throw new Error('未识别到网站编辑器:' + name + '。请刷新页面后重试。');
}
if (!win.imgUpload?.settings || typeof win.updateImageList !== 'function') throw new Error('网站图片上传组件尚未就绪,请刷新后重试。');
if (Number(win.allowbbcode) === 0 || Number(win.allowimgcode) === 0) throw new Error('当前版块或账号不支持正文格式/图片。');
}
async function uploadImage(blob, path, settings, signal) {
signal?.throwIfAborted();
const url = new URL(settings.upload_url, win.location.href);
if (url.origin !== win.location.origin || url.pathname !== '/misc.php' || url.searchParams.get('operation') !== 'upload') {
throw new Error('上传地址与预期不符,已停止。');
}
const hash = Array.from(new Uint8Array(await win.crypto.subtle.digest('SHA-256', await blob.arrayBuffer())), n => n.toString(16).padStart(2, '0')).join('');
const key = url.href + ':' + settings.post_params.uid + ':' + hash;
if (cache.has(key)) { notify('upload', '复用已上传图片:' + path); return cache.get(key); }
const data = new FormData();
for (const [key, value] of Object.entries(settings.post_params)) data.append(key, String(value));
data.set('type', 'image');
data.set('filetype', blob.type || (/\.png$/i.test(path) ? 'image/png' : /\.gif$/i.test(path) ? 'image/gif' : 'image/jpeg'));
data.append(settings.file_post_name || 'Filedata', blob, path.split('/').pop());
const control = new AbortController(), timer = setTimeout(() => control.abort(), 90000);
try {
signal?.throwIfAborted();
signal?.addEventListener('abort', () => control.abort(), { once: true });
const response = await win.fetch(url.href, { method: 'POST', body: data, credentials: 'same-origin', redirect: 'error', signal: control.signal });
if (!response.ok) throw new Error('图片上传请求失败:HTTP ' + response.status);
const aid = parseAid(await response.text());
cache.set(key, aid);
return aid;
} catch (error) {
if (error.name === 'AbortError') throw new Error('上传超时;该图片可能已到达服务器,请检查网站图片列表后再重试。');
throw error;
} finally { clearTimeout(timer); }
}
async function registerImages(aids, signal) {
signal?.throwIfAborted();
if (!aids.length) return;
const ids = new Set(aids);
doc.querySelectorAll('#imgattachlist [id^="image_td_"],#imgattachlist [id^="image_"]').forEach(el => {
const id = el.id.match(/^(?:image_td_|image_)(\d+)$/)?.[1]; if (id) ids.add(id);
});
const ready = () => aids.every(id => doc.getElementById('image_' + id));
if (ready()) return;
// Use the site's own list loader, which registers attachment form fields too.
win.updateImageList(1, Array.from(ids).join('|'));
await new Promise((resolve, reject) => {
const timer = setTimeout(() => { observer.disconnect(); reject(new Error('图片已上传,但网站图片列表加载超时。重试会复用本页已上传图片。')); }, 25000);
signal?.addEventListener('abort', () => { clearTimeout(timer); observer.disconnect(); reject(new Error('流程已终止')); }, { once: true });
const check = () => { if (ready()) { clearTimeout(timer); observer.disconnect(); resolve(); } };
const observer = new MutationObserver(check);
observer.observe(doc.body, { childList: true, subtree: true }); check();
});
}
function setEditor(title, bbcode) {
win.switchEditor(0);
if (Number(win.wysiwyg) !== 0) throw new Error('无法切换编辑器为纯文本模式。');
const textarea = doc.querySelector('#e_textarea');
textarea.value = bbcode;
doc.querySelector('#subject').value = title;
for (const el of [textarea, doc.querySelector('#subject')]) {
el.dispatchEvent(new Event('input', { bubbles: true })); el.dispatchEvent(new Event('change', { bubbles: true }));
}
// Let Discuz generate the WYSIWYG content with native attachment metadata.
win.switchEditor(1);
const switcher = doc.querySelector('#e_switchercheck');
if (switcher) switcher.checked = Number(win.wysiwyg) === 0;
}
async function importPrepared(options = {}) {
const { signal, automatic = false } = options;
signal?.throwIfAborted();
if (busy) throw new Error('正在导入,请勿重复点击。');
if (!article) throw new Error('请先选择文章目录或文章包。');
assertEditor();
for (const id of ['bbcodeoff', 'htmlon']) {
if (doc.getElementById(id)?.checked) throw new Error('请先关闭“禁用 Discuz! 代码”和“HTML 代码”,再导入。');
}
const before = snapshot();
if ((before.title.trim() || before.text.replace(/<[^>]*>/g, '').trim() || /<img\b/i.test(before.text)) &&
(automatic ? (() => { throw new Error('编辑器已有内容,请人工处理旧稿后重试'); })() : !win.confirm('当前编辑器已有内容。导入将替换标题和正文,是否继续?'))) return { cancelled: true };
const selected = article;
busy = true;
const controls = ['choose', 'package', 'import', 'md'].map($);
controls.forEach(el => { el.disabled = true; });
// Prevent accidental submission of a partly imported article.
const blockSubmit = event => { event.preventDefault(); event.stopImmediatePropagation(); notify('busy', '正在导入,请等待完成后再操作发布。'); };
const form = doc.querySelector('#postform');
form.addEventListener('submit', blockSubmit, true);
const submitButtons = Array.from(form.querySelectorAll('button[type="submit"],button[name],input[type="submit"]'));
const previousDisabled = submitButtons.map(el => el.disabled);
submitButtons.forEach(el => { el.disabled = true; });
let committed = false;
try {
const aids = new Map();
const settings = win.imgUpload.settings;
const max = Math.min(MAX_IMAGE_SIZE, Number(settings.file_size_limit) * 1024 || MAX_IMAGE_SIZE);
for (const [path, blob] of selected.images) if (blob.size >= max) throw new Error('配图超过当前网站限制:' + path);
let index = 0;
for (const [path, blob] of selected.images) {
notify('upload', `上传 ${++index}/${selected.images.size}:${path}`, { completed: index - 1, total: selected.images.size });
aids.set(path, await uploadImage(blob, path, settings, signal));
}
notify('register', '正在加载网站图片列表…');
await registerImages(Array.from(new Set(aids.values())), signal);
signal?.throwIfAborted();
const now = snapshot();
if ((now.title !== before.title || now.text !== before.text) && (automatic ? (() => { throw new Error('上传期间编辑器被修改,已保留当前内容'); })() : !win.confirm('上传期间你修改了编辑器。是否用导入文章替换最新内容?'))) {
notify('cancelled', '已保留当前正文;图片已上传,可稍后再次导入。'); return { cancelled: true };
}
const backup = snapshot();
const bbcode = selected.render(aids);
// Preserve a local, downloadable snapshot before changing title or body.
savedBackup = { title: backup.title, content: backup.text, mode: backup.mode, createdAt: new Date().toISOString() };
try {
for (const id of ['bbcodeoff', 'htmlon']) {
const el = doc.getElementById(id);
if (el && id === 'bbcodeoff' && el.checked) throw new Error('请先取消“禁用 Discuz! 代码”,再重试。');
if (el && id === 'htmlon' && el.checked) throw new Error('请先关闭“HTML 代码”,再重试。');
}
signal?.throwIfAborted();
setEditor(selected.title, bbcode);
const raw = win.getEditorContents();
if (Number(win.wysiwyg) === 1) {
for (const id of new Set(aids.values())) if (!raw.includes('attachimg_' + id)) throw new Error('富文本图片转换未通过检查。');
} else if (raw !== bbcode) throw new Error('正文填写校验失败。');
if (doc.querySelector('#subject').value !== selected.title) throw new Error('标题填写校验失败。');
committed = true;
} catch (error) {
// Restore the exact editor representation saved immediately before filling.
win.switchEditor(backup.mode);
if (backup.mode) win.editdoc.body.innerHTML = backup.text;
else doc.querySelector('#e_textarea').value = backup.text;
doc.querySelector('#subject').value = backup.title;
const switcher = doc.querySelector('#e_switchercheck'); if (switcher) switcher.checked = !backup.mode;
throw error;
}
lastResult = { title: selected.title, images: Object.fromEntries(aids), completedAt: new Date().toISOString() };
notify('complete', `已填写标题、正文和 ${aids.size} 张配图。请检查排版,随后手动发表。`);
return lastResult;
} catch (error) {
notify('error', error.message + (committed ? '' : ' 标题和正文未替换;已上传的图片会保留。'));
throw error;
} finally {
busy = false; form.removeEventListener('submit', blockSubmit, true);
submitButtons.forEach((el, index) => { el.disabled = previousDisabled[index]; });
controls.forEach(el => { el.disabled = false; });
$('import').disabled = !article;
}
}
function downloadJSON(value, filename) {
const url = URL.createObjectURL(new Blob([JSON.stringify(value, null, 2)], { type: 'application/json' }));
const link = doc.createElement('a'); link.href = url; link.download = filename; link.click();
setTimeout(() => URL.revokeObjectURL(url), 1000);
}
// Public v1 interface. Staging is read-only; importPrepared is the explicit upload/fill step.
// No postMessage listener, localhost service, publishing endpoint or saved credentials.
const api = Object.freeze({ version: VERSION, schemaVersion: 1, stageFiles, stagePackage,
getState: summary, openPicker: () => { if (!busy) $('folder').click(); }, importPrepared,
getBackup: () => savedBackup });
win.DWW11Importer = api;
$('choose').onclick = () => { $('folder').value = ''; $('folder').click(); };
$('package').onclick = () => { $('json').value = ''; $('json').click(); };
const handleError = error => notify('error', error.message || String(error));
$('folder').onchange = () => stageFiles($('folder').files).catch(handleError);
$('json').onchange = async () => {
try { const file = $('json').files[0]; if (file) await stagePackage(JSON.parse(await file.text())); }
catch (error) { handleError(error); }
};
$('md').onchange = () => {
const files = candidates, path = $('md').value;
if (files && path) stageFiles(files, { markdownPath: path }).catch(handleError);
};
$('import').onclick = () => importPrepared().catch(handleError);
$('diagnostics').onclick = () => downloadJSON({ ...summary(), logs }, 'dww11-import-diagnostics.json');
win.addEventListener('pagehide', releasePreviews, { once: true });
}
return { VERSION, normalizePath, imagePath, readLink, markdownToBBCode, compile, prepareFiles, preparePackage, parseAid, mount };
});
// Extracted from dww11-wechat-oneclick.user.js v1.0.9; package source is authoritative.
(() => {
'use strict';
const cleanText = value => String(value || '').replace(/\s+/g, ' ').trim();
function flattenParagraphs(body) {
const selector = 'p,div,section,article,figure,figcaption,h1,h2,h3,h4,h5,h6';
const marker = () => {
const br = body.ownerDocument.createElement('br');
br.dataset.dww11Boundary = '1'; return br;
};
for (const block of Array.from(body.querySelectorAll(selector)).reverse()) {
if (block.closest('table,ul,ol,blockquote')) continue;
block.replaceWith(marker(), ...block.childNodes, marker());
}
let previous = false, pending = [];
function finish() {
const actual = pending.filter(br => !br.hasAttribute('data-dww11-boundary'));
let retained = false;
for (const br of pending) {
if (!br.hasAttribute('data-dww11-boundary')) continue;
if (previous && !actual.length && !retained) { br.removeAttribute('data-dww11-boundary'); retained = true; }
else br.remove();
}
pending = [];
}
const walker = body.ownerDocument.createTreeWalker(body, NodeFilter.SHOW_ELEMENT | NodeFilter.SHOW_TEXT);
const nodes = []; while (walker.nextNode()) nodes.push(walker.currentNode);
for (const node of nodes) {
if (node.nodeType === 1 && node.tagName === 'BR') pending.push(node);
else if ((node.nodeType === 3 && node.textContent.trim()) || (node.nodeType === 1 && ['IMG', 'VIDEO', 'HR', 'IFRAME'].includes(node.tagName))) {
finish(); previous = true;
}
}
// 末尾不保留由容器边界制造的额外换行。
for (const br of pending) if (br.hasAttribute('data-dww11-boundary')) br.remove();
}
function spaceImages(body) {
let previous = null, gapNodes = [], blankLines = 0;
const changes = [];
function content(kind, node) {
if (previous && previous.kind !== kind) {
let anchor = node;
while (anchor.parentElement && anchor.parentElement !== body &&
!['TD', 'TH', 'LI', 'BLOCKQUOTE'].includes(anchor.parentElement.tagName) && !anchor.parentElement.contains(previous.node)) anchor = anchor.parentElement;
changes.push({ anchor, nodes: gapNodes.slice(), count: Math.max(2, blankLines) });
}
previous = { kind, node }; gapNodes = []; blankLines = 0;
}
function visit(node) {
if (node.nodeType === 3) {
if (node.textContent.trim()) content('text', node); else gapNodes.push(node);
return;
}
if (node.nodeType !== 1 || ['SCRIPT', 'STYLE', 'NOSCRIPT'].includes(node.tagName)) return;
if (node.tagName === 'BR') { gapNodes.push(node); blankLines++; return; }
if (node.tagName === 'IMG') { content('image', node); return; }
for (const child of node.childNodes) visit(child);
}
for (const child of body.childNodes) visit(child);
for (const { anchor, nodes, count } of changes) {
for (const node of nodes) node.remove();
const fragment = body.ownerDocument.createDocumentFragment();
for (let i = 0; i < count; i++) fragment.append(body.ownerDocument.createElement('br'));
anchor.parentNode.insertBefore(fragment, anchor);
}
}
function formatBody(body, author, sourceText = null) {
const doc = body.ownerDocument;
body.querySelectorAll('[data-dww11-wx-source="1"]').forEach(el => el.remove());
// Discuz 会把内联 color 值直接转为 [color=…]。
// !important 不能出现在该值中;颜色和字号只用原生 FONT 属性。
for (const el of [body, ...body.querySelectorAll('*')]) {
if (['SCRIPT', 'STYLE'].includes(el.tagName)) continue;
for (const property of ['font', 'font-size', 'color', '-webkit-text-fill-color']) {
el.style.removeProperty(property);
}
// 其他排版属性也不保留 CSS 优先级,避免它泄漏进论坛标签。
for (const property of Array.from(el.style)) {
if (el.style.getPropertyPriority(property)) {
el.style.setProperty(property, el.style.getPropertyValue(property));
}
}
if (!el.style.cssText) el.removeAttribute('style');
}
// 去掉采集器或旧版脚本生成的嵌套 FONT,保留其中的内容和图片。
for (const font of Array.from(body.querySelectorAll('font'))) {
font.replaceWith(...font.childNodes);
}
flattenParagraphs(body);
spaceImages(body);
// 所有正文内容和换行放在同一 FONT 中。不能使用只有 BR 的空 FONT:
// 论坛转换器可能清除空格式标签,导致空行退回默认1号或直接丢失。
const articleFont = doc.createElement('font');
articleFont.size = '4'; articleFont.color = '#000000';
articleFont.dataset.dww11WxBody = '1';
articleFont.append(...body.childNodes);
body.append(articleFont);
// A 自身的浏览器默认蓝色会覆盖外层继承色,链接文字另设黑色。
for (const link of articleFont.querySelectorAll('a')) {
const linkFont = doc.createElement('font');
linkFont.size = '4'; linkFont.color = '#000000';
linkFont.append(...link.childNodes); link.append(linkFont);
}
// 只用于编辑器图片适配;帖子展示仍交给网站原有图片/附件规则。
for (const img of articleFont.querySelectorAll('img')) {
img.style.maxWidth = '100%'; img.style.height = 'auto';
}
const source = doc.createElement('p');
source.dataset.dww11WxSource = '1';
const font = doc.createElement('font');
font.size = '1'; font.color = '#c0c0c0';
const lines = (sourceText ?? `来源:${author},如有侵权,请联系删除。`).split('\n');
lines.forEach((line, index) => {
if (index) font.append(doc.createElement('br'));
font.append(doc.createTextNode(line));
});
source.append(font); body.append(source);
}
function existingSource(body) {
// 按段落和 BR 读取末尾逻辑行,兼容嵌套 FONT、纯文本及末尾空段落。
const lines = [];
let parts = [];
function flush() {
const text = cleanText(parts.map(part => part.node.textContent.slice(part.start, part.end)).join(''));
if (text) lines.push({ text, parts });
parts = [];
}
function visit(node) {
if (node.nodeType === 3) {
// 保留文本中的显式换行,并记录精确位置,避免删除同容器中的正文。
const value = node.textContent;
let start = 0;
for (let i = 0; i <= value.length; i++) {
if (i !== value.length && value[i] !== '\n') continue;
// HTML 缩进换行通常只是空白;只有后续是来源或版权行才分行。
if (i < value.length && !/^\s*(?:(?:来源|文章来源|本文来源|图文来源|文字来源|作者|作者信息|来源作者|转自|转载自|出处)\s*[::]|如有侵权|若有侵权|侵权请|转载请|版权归)/.test(value.slice(i + 1))) continue;
if (i > start) parts.push({ node, start, end: i });
if (i < value.length) flush();
start = i + 1;
}
return;
}
if (node.nodeType !== 1 || ['SCRIPT', 'STYLE', 'NOSCRIPT'].includes(node.tagName)) return;
if (node.tagName === 'BR') { flush(); return; }
if (['IMG', 'VIDEO', 'AUDIO', 'IFRAME', 'HR'].includes(node.tagName)) {
flush(); lines.push({ text: '\ufffc', parts: [] }); return;
}
const block = /^(P|DIV|SECTION|ARTICLE|FIGURE|FIGCAPTION|H[1-6]|LI|TR|BLOCKQUOTE|PRE)$/.test(node.tagName);
if (block) flush();
for (const child of node.childNodes) visit(child);
if (block) flush();
}
for (const child of body.childNodes) visit(child);
flush();
let first = -1;
for (let i = lines.length - 1; i >= 0; i--) {
if (/^(?:来源|文章来源|本文来源|图文来源|文字来源|作者|作者信息|来源作者|转自|转载自|出处)\s*[::]\s*\S/.test(lines[i].text)) {
first = i;
} else if (!/^(?:如有侵权|若有侵权|侵权请|转载请|版权归)/.test(lines[i].text)) {
break;
}
}
if (first < 0) return null;
const start = lines[first].parts[0];
const range = body.ownerDocument.createRange();
range.setStart(start.node, start.start);
range.setEnd(body, body.childNodes.length);
return {
text: lines.slice(first).map(line => line.text).join('\n'),
remove: () => range.deleteContents()
};
}
window.DWW11Formatter = Object.freeze({
format(expectedSource) {
if (Number(window.wysiwyg) !== 1) throw new Error('请切换到可视化编辑器后排版');
const body = document.querySelector('#e_iframe')?.contentDocument?.body;
if (!body) throw new Error('编辑器未就绪');
const footer = existingSource(body);
const expected = `来源:${expectedSource},如有侵权,请联系删除。`;
if (!footer || footer.text !== expected) throw new Error('文末来源与文章包不一致,请人工检查');
const images = [...body.querySelectorAll('img')].map(x => x.id + ':' + x.getAttribute('src'));
const backup = body.innerHTML;
try {
footer.remove();
formatBody(body, expectedSource, expected);
const after = [...body.querySelectorAll('img')].map(x => x.id + ':' + x.getAttribute('src'));
if (JSON.stringify(images) !== JSON.stringify(after)) throw new Error('排版后图片标识或顺序改变');
if (body.querySelectorAll('[data-dww11-wx-source="1"]').length !== 1) throw new Error('来源尾注数量异常');
body.dispatchEvent(new Event('input', {bubbles:true}));
return {images: after, source: expected};
} catch (error) { body.innerHTML = backup; throw error; }
}
});
})();
/* Page-side orchestrator. This file never clicks or submits the publication form. */
(() => {
'use strict';
if (window.DWW11NewsRunner) return;
window.DWW11NewsRunner = true;
const SLOT='auto-news-current';
const owner=sessionStorage.getItem('auto-news-owner') || crypto.randomUUID();
sessionStorage.setItem('auto-news-owner',owner);
let run=null, working=false, halted=true, aborter=null, active=null, tickBusy=false;
const pending=new Map();
function rpc(route,body={}) {
return new Promise((resolve,reject)=>{
const id=crypto.randomUUID();
const timer=setTimeout(()=>{pending.delete(id);reject(new Error('本地连接无响应'));},65000);
pending.set(id,{resolve,reject,timer});
document.dispatchEvent(new CustomEvent('dww11:local-request',{detail:JSON.stringify({id,route,body:{owner,...body}})}));
});
}
document.addEventListener('dww11:local-response',event=>{
let data;try{data=JSON.parse(event.detail);}catch{return;}
const waiter=pending.get(data.id);if(!waiter)return;
clearTimeout(waiter.timer);pending.delete(data.id);
data.error?waiter.reject(new Error(data.error)):waiter.resolve(data.value);
});
const status=message=>document.dispatchEvent(new CustomEvent('dww11:connection-status',{detail:message}));
document.addEventListener('dww11:auto-stop',()=>{halted=true;aborter?.abort(new Error('流程已终止'));});
function account() {
const element=document.querySelector('#mumucms_username');
if(!element)return null;
const name=[...element.childNodes].filter(n=>n.nodeType===Node.TEXT_NODE).map(n=>n.textContent).join('').trim();
const links=[element,...element.querySelectorAll('a')].filter(e=>e.href);
let uid=null;
for(const link of links) {
const href=link.href;
uid=new URL(href,location.href).searchParams.get('uid') || href.match(/space-uid-(\d+)/)?.[1];
if(uid)break;
}
return uid&&name?{uid,name}:null;
}
function extension() {
try{return JSON.parse(document.querySelector('#dww11-extension-panel')?.getAttribute('data-auto-news-state') || '{}');}catch{return {};}
}
function assertLive(epoch) {
if(halted||!run?.running||run.epoch!==epoch)throw new Error('流程已暂停或终止');
if(active?.account?.uid!==account()?.uid)throw new Error('当前账号与任务不一致');
}
function editorSnapshot() {
const body=document.querySelector('#e_iframe')?.contentDocument?.body;
if(!body)throw new Error('找不到可视化编辑器');
return {title:document.querySelector('#subject')?.value || '',
images:[...body.querySelectorAll('img')].map(x=>({id:x.id,src:x.getAttribute('src')})),
// Snapshot only final submitted content; never includes hidden tokens or the full form.
html:body.innerHTML.slice(0,500000), text:body.innerText.slice(0,100000)};
}
function saveLocal(value) {sessionStorage.setItem(SLOT,JSON.stringify(value));}
function loadLocal(){try{return JSON.parse(sessionStorage.getItem(SLOT)||'null');}catch{return null;}}
async function event(state,extra={}) {return rpc('event',{id:active.id,state,epoch:run.epoch,...extra});}
async function fail(error) {
status(error.message);
if(active)try{await event('needs_attention',{error:error.message});}catch{}
halted=true;aborter?.abort();document.dispatchEvent(new CustomEvent('dww11:auto-stop'));
}
function editorPage(job) {
const u=new URL(location.href);
return u.searchParams.get('mod')==='post'&&u.searchParams.get('action')==='newthread'&&u.searchParams.get('fid')===String(job.fid);
}
async function prepare(job) {
working=true;active=job;aborter=new AbortController();const epoch=run.epoch;
try {
assertLive(epoch);
if(!editorPage(job)) {
saveLocal({id:job.id});
location.assign(`https://www.dww11.com/forum.php?mod=post&action=newthread&fid=${job.fid}`);return;
}
if(!window.DWW11Importer||!window.DWW11Formatter)throw new Error('导入或排版模块尚未就绪,请检查安装');
status('正在检查文章包…');
const pack=await rpc('package',{id:job.id});assertLive(epoch);
const prepared=await window.DWW11Importer.stagePackage(pack);assertLive(epoch);
if(prepared.warnings?.length)throw new Error('文章需要人工处理:'+prepared.warnings.join(';'));
await event('importing');assertLive(epoch);status('上传图片并填写文章…');
const result=await window.DWW11Importer.importPrepared({automatic:true,signal:aborter.signal});assertLive(epoch);
if(result?.cancelled)throw new Error('导入已取消');
await event('formatting');assertLive(epoch);
window.DWW11Formatter.format(job.source);assertLive(epoch);
const snapshot=editorSnapshot();
if(snapshot.title!==job.title)throw new Error('标题校验失败');
const aids=new Set(Object.values(result.images));
for(const aid of aids) if(!snapshot.images.some(x=>x.id==='attachimg_'+aid))throw new Error('配图附件缺失:'+aid);
await event('awaiting_user_review',{snapshot});assertLive(epoch);
active={...job,state:'awaiting_user_review'};saveLocal({id:job.id});
status('已完成排版,请核对内容并亲手点击“发表帖子”。');
}catch(error){await fail(error);}finally{working=false;}
}
// Record only trusted user activation. Never replay a click, call submit(), or send a post request.
document.addEventListener('click',event=>{
if(!event.isTrusted||!event.target.closest?.('#postsubmit')||active?.state!=='awaiting_user_review')return;
try {
const snapshot=editorSnapshot();
const local={id:active.id,snapshot,clickedAt:Date.now(),observed:false};saveLocal(local);
rpc('event',{id:active.id,state:'user_submitting',epoch:run?.epoch,snapshot})
.then(()=>{const latest=loadLocal();if(latest?.clickedAt===local.clickedAt)saveLocal({...local,observed:true});})
.catch(error=>status('提交记录待恢复:'+error.message));
active.state='user_submitting';status('已观察到人工点击,等待结果页核对。');
}catch(error){status('提交快照记录失败:'+error.message);}
},true);
async function observeResult(job) {
const local=loadLocal();
if(local?.id===job.id&&local.snapshot&&job.state==='awaiting_user_review') {
await rpc('event',{id:job.id,state:'user_submitting',epoch:run.epoch,snapshot:local.snapshot});
job.state='user_submitting';
}
if(job.state==='user_submitting') {
// Result selectors intentionally await user-provided HTML; URL alone is not proof.
if(!editorPage(job) || (local?.clickedAt && Date.now()-local.clickedAt>45000)) {
await rpc('event',{id:job.id,state:'submit_unknown',error:'等待结果页 HTML 适配;候选页面:'+location.href});
await rpc('stop');
status('已记录候选结果页;尚不能自动核对。流程已暂停,请补充结果页 HTML。');
}
}else if(job.state==='submit_unknown')status('发布结果待核对;不会重发、计数或换号。');
}
async function switchAccount() {
if(working||halted)return;
if(location.origin!=='https://www.dww11.com'||location.pathname!=='/') {
location.assign('https://www.dww11.com/');return;
}
const ext=extension(),button=document.querySelector('#dww11-extension-next');
if(!ext.ready||!ext.idle||!button||button.disabled) {status('请安装并重新加载带联动接口的换号插件');return;}
await rpc('switch-begin');
if(halted||!run?.running)return;
button.click();status('正在换号;等待插件完成并核对新账号');
}
async function tick() {
if(tickBusy)return;tickBusy=true;
try {
run=await rpc('heartbeat',{account:account(),url:location.href,extension:extension()});
halted=!run.running;
if(!run.running) {aborter?.abort();document.dispatchEvent(new CustomEvent('dww11:auto-stop'));}
if(working)return;
// Passive result recording is permitted after Stop, but no new business actions.
if(run.active) {
const local=loadLocal();
if(local?.id===run.active&&local.snapshot&&!run.running)status('流程已暂停;提交结果仍需核对');
}
if(!run.running) {
if(run.active && run.owner===owner) {const saved=await rpc('current');if(saved.job) {active=saved.job;await observeResult(saved.job);}}
status(run.reason || '流程已暂停');return;
}
const next=await rpc('next',{account:account(),extension:extension()});run=next.run;
if(next.job)active=next.job;
if(next.action==='prepare'||(next.action==='resume'&&next.job.state==='preparing'))void prepare(next.job);
else if(next.action==='resume') {
await observeResult(next.job);
if(next.job.state==='awaiting_user_review') {
if(!editorPage(next.job)||!document.querySelector('#subject')?.value) await fail(new Error('待审稿页面已丢失,请人工核对旧稿,不自动重新导入'));
else status('等待你亲手发表;程序不再修改编辑器');
}
else if(['importing','formatting'].includes(next.job.state))await fail(new Error('导入阶段发生中断,请检查编辑器和已上传图片后处理任务'));
else if(next.job.state==='needs_attention')status(next.job.error);
}else if(next.action==='switch')await switchAccount();
else if(next.action==='switch_wait')status('等待换号结果;若插件已暂停,请人工处理后继续');
else if(next.action==='login')status('未能读取当前账号 UID,请登录并确认账号元素');
else if(next.action==='other_tab')status('另一个标签持有任务;如需接管请先暂停');
else if(next.action==='interval')status('等待设定的处理间隔');
else status(run.reason || '等待任务');
}catch(error){status(error.message);halted=true;aborter?.abort();document.dispatchEvent(new CustomEvent('dww11:auto-stop'));}
finally{tickBusy=false;}
}
const takeover=document.createElement('button');takeover.type='button';takeover.textContent='接管已暂停任务';
takeover.style.cssText='position:fixed;left:16px;top:16px;z-index:2147483646;padding:8px;background:#fff;border:1px solid #aab;border-radius:6px';
takeover.onclick=async()=>{try{await rpc('takeover');status('已接管;核对当前页面后在工作台继续');}catch(error){status(error.message);}};
document.body.append(takeover);
setInterval(tick,2000);tick();
})();
+397
View File
@@ -0,0 +1,397 @@
// ==UserScript==
// @name 广谈大湾网 · 微信文章一键导入并排版
// @namespace local.dww11.wechat-oneclick
// @version 1.0.9
// @description 使用网站现有微信采集器,修正标题,统一正文4号黑色,追加1号银色来源;不点击保存或发布。
// @match https://www.dww11.com/forum.php*
// @match https://dww11.com/forum.php*
// @grant GM_xmlhttpRequest
// @grant unsafeWindow
// @connect mp.weixin.qq.com
// @run-at document-idle
// @noframes
// ==/UserScript==
(() => {
'use strict';
const ID = 'dww11-wx-oneclick';
const FORMAT_ID = 'dww11-format-oneclick';
let busy = false;
function setBusy(value) {
busy = value;
for (const id of [ID, FORMAT_ID]) {
const button = document.getElementById(id);
if (button) button.disabled = value;
}
}
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
const visible = el => el && el.getClientRects().length && getComputedStyle(el).display !== 'none';
async function waitFor(fn, timeout = 10000) {
const deadline = Date.now() + timeout;
while (Date.now() < deadline) {
const result = fn();
if (result) return result;
await sleep(100);
}
throw new Error('等待编辑器或采集窗口超时,请关闭遮挡窗口后重试。');
}
function normalizeURL(raw) {
const url = new URL(raw.trim().replace(/[\u200b-\u200d\ufeff]/g, ''));
if (url.protocol !== 'https:' || url.hostname !== 'mp.weixin.qq.com' || url.port || url.username || url.password || !/^\/(s(?:\/|$)|s$)/.test(url.pathname)) {
throw new Error('请输入 https://mp.weixin.qq.com/s/… 或 /s?… 微信文章地址。');
}
url.hash = '';
return url.href;
}
function cleanText(value) {
return String(value || '').replace(/\s+/g, ' ').trim();
}
function metadata(html) {
const doc = new DOMParser().parseFromString(html, 'text/html');
const text = selector => cleanText(doc.querySelector(selector)?.textContent);
const meta = selector => cleanText(doc.querySelector(selector)?.getAttribute('content'));
// 与下载组件一致,先确认是真正的文章,避免把验证页标题当成文章。
if (!doc.querySelector('#js_content')) throw new Error('未找到微信文章正文,可能需要验证、登录,或文章已失效。');
const title = text('#activity-name .js_title_inner') || text('#activity-name') || meta('meta[property="og:title"]') || meta('meta[name="twitter:title"]') || text('h1') || text('title');
// 不读取 rich_media_meta_text 容器:它可能包含重复的隐藏作者,或实际上是日期/IP。
// 来源使用蓝色链接 #js_name 的公众号名称,不使用灰色署名作者。
const author = text('#js_name');
if (!title || !author) throw new Error('微信未返回完整标题和公众号名称,可能需要验证或登录。');
return { title, author };
}
function fetchMetadata(url) {
return new Promise((resolve, reject) => {
GM_xmlhttpRequest({
method: 'GET', url, timeout: 25000, anonymous: true,
onload(response) {
try {
if (response.status !== 200) throw new Error('微信返回状态 ' + response.status);
if (response.finalUrl && new URL(response.finalUrl).hostname !== 'mp.weixin.qq.com') throw new Error('微信地址发生了非预期跳转。');
resolve(metadata(response.responseText));
} catch (error) { reject(error); }
},
onerror: () => reject(new Error('无法读取微信文章。')),
ontimeout: () => reject(new Error('读取微信文章超时。'))
});
});
}
async function getMetadata(url) {
try { return await fetchMetadata(url); }
catch (error) {
const title = prompt(error.message + '\n请从源文章复制完整标题(取消则停止导入):');
if (title === null) return null;
const author = prompt('请填写源文章蓝色链接处的公众号名称(取消则停止导入):');
if (author === null) return null;
if (!cleanText(title) || !cleanText(author)) throw new Error('标题和作者不能为空。');
return { title: cleanText(title), author: cleanText(author) };
}
}
function editorBody() {
return document.querySelector('#e_iframe')?.contentDocument?.body;
}
async function ensureVisualEditor() {
const checkbox = document.querySelector('#e_switchercheck');
if (checkbox?.checked) checkbox.click();
await waitFor(() => visible(document.querySelector('#e_iframe')) && editorBody());
}
// 只监听本次现有采集器的响应,不重写其请求或图片附件处理。
function watchImport(jq, url) {
let timer, resolveResult, rejectResult;
const result = new Promise((resolve, reject) => { resolveResult = resolve; rejectResult = reject; });
const events = jq(document);
const cleanup = () => { clearTimeout(timer); events.off('.dww11wx'); };
const finish = error => { cleanup(); error ? rejectResult(error) : resolveResult(); };
events.on('ajaxComplete.dww11wx', (_event, xhr, settings) => {
let requestURL, sentURL;
try {
requestURL = new URL(settings.url, location.href);
sentURL = typeof settings.data === 'string' ? new URLSearchParams(settings.data).get('url') : settings.data?.url;
} catch { return; }
if (requestURL.origin !== location.origin || requestURL.searchParams.get('id') !== 'wxcj:cj' || sentURL !== url) return;
const parts = String(xhr.responseText || '').split('----wxcj----');
if (xhr.status < 200 || xhr.status >= 300 || parts.length < 4 || !parts[2]?.trim()) {
finish(new Error('网站采集失败或返回内容不完整。未执行排版,请检查采集提示。'));
return;
}
finish();
});
timer = setTimeout(() => finish(new Error('网站采集超过90秒,未执行排版。采集器仍可能在后台返回,请勿连续重复导入。')), 90000);
return { result, cleanup };
}
// 将微信 P/SECTION 的隐式换行改成真实 BR,避免论坛忽略 SECTION 边界。
function flattenParagraphs(body) {
const selector = 'p,div,section,article,figure,figcaption,h1,h2,h3,h4,h5,h6';
const marker = () => {
const br = body.ownerDocument.createElement('br');
br.dataset.dww11Boundary = '1'; return br;
};
for (const block of Array.from(body.querySelectorAll(selector)).reverse()) {
if (block.closest('table,ul,ol,blockquote')) continue;
block.replaceWith(marker(), ...block.childNodes, marker());
}
let previous = false, pending = [];
function finish() {
const actual = pending.filter(br => !br.hasAttribute('data-dww11-boundary'));
let retained = false;
for (const br of pending) {
if (!br.hasAttribute('data-dww11-boundary')) continue;
if (previous && !actual.length && !retained) { br.removeAttribute('data-dww11-boundary'); retained = true; }
else br.remove();
}
pending = [];
}
const walker = body.ownerDocument.createTreeWalker(body, NodeFilter.SHOW_ELEMENT | NodeFilter.SHOW_TEXT);
const nodes = []; while (walker.nextNode()) nodes.push(walker.currentNode);
for (const node of nodes) {
if (node.nodeType === 1 && node.tagName === 'BR') pending.push(node);
else if ((node.nodeType === 3 && node.textContent.trim()) || (node.nodeType === 1 && ['IMG', 'VIDEO', 'HR', 'IFRAME'].includes(node.tagName))) {
finish(); previous = true;
}
}
// 末尾不保留由容器边界制造的额外换行。
for (const br of pending) if (br.hasAttribute('data-dww11-boundary')) br.remove();
}
function spaceImages(body) {
let previous = null, gapNodes = [], blankLines = 0;
const changes = [];
function content(kind, node) {
if (previous && previous.kind !== kind) {
let anchor = node;
while (anchor.parentElement && anchor.parentElement !== body &&
!['TD', 'TH', 'LI', 'BLOCKQUOTE'].includes(anchor.parentElement.tagName) && !anchor.parentElement.contains(previous.node)) anchor = anchor.parentElement;
changes.push({ anchor, nodes: gapNodes.slice(), count: Math.max(2, blankLines) });
}
previous = { kind, node }; gapNodes = []; blankLines = 0;
}
function visit(node) {
if (node.nodeType === 3) {
if (node.textContent.trim()) content('text', node); else gapNodes.push(node);
return;
}
if (node.nodeType !== 1 || ['SCRIPT', 'STYLE', 'NOSCRIPT'].includes(node.tagName)) return;
if (node.tagName === 'BR') { gapNodes.push(node); blankLines++; return; }
if (node.tagName === 'IMG') { content('image', node); return; }
for (const child of node.childNodes) visit(child);
}
for (const child of body.childNodes) visit(child);
for (const { anchor, nodes, count } of changes) {
for (const node of nodes) node.remove();
const fragment = body.ownerDocument.createDocumentFragment();
for (let i = 0; i < count; i++) fragment.append(body.ownerDocument.createElement('br'));
anchor.parentNode.insertBefore(fragment, anchor);
}
}
function formatBody(body, author, sourceText = null) {
const doc = body.ownerDocument;
body.querySelectorAll('[data-dww11-wx-source="1"]').forEach(el => el.remove());
// Discuz 会把内联 color 值直接转为 [color=…]。
// !important 不能出现在该值中;颜色和字号只用原生 FONT 属性。
for (const el of [body, ...body.querySelectorAll('*')]) {
if (['SCRIPT', 'STYLE'].includes(el.tagName)) continue;
for (const property of ['font', 'font-size', 'color', '-webkit-text-fill-color']) {
el.style.removeProperty(property);
}
// 其他排版属性也不保留 CSS 优先级,避免它泄漏进论坛标签。
for (const property of Array.from(el.style)) {
if (el.style.getPropertyPriority(property)) {
el.style.setProperty(property, el.style.getPropertyValue(property));
}
}
if (!el.style.cssText) el.removeAttribute('style');
}
// 去掉采集器或旧版脚本生成的嵌套 FONT,保留其中的内容和图片。
for (const font of Array.from(body.querySelectorAll('font'))) {
font.replaceWith(...font.childNodes);
}
flattenParagraphs(body);
spaceImages(body);
// 所有正文内容和换行放在同一 FONT 中。不能使用只有 BR 的空 FONT:
// 论坛转换器可能清除空格式标签,导致空行退回默认1号或直接丢失。
const articleFont = doc.createElement('font');
articleFont.size = '4'; articleFont.color = '#000000';
articleFont.dataset.dww11WxBody = '1';
articleFont.append(...body.childNodes);
body.append(articleFont);
// A 自身的浏览器默认蓝色会覆盖外层继承色,链接文字另设黑色。
for (const link of articleFont.querySelectorAll('a')) {
const linkFont = doc.createElement('font');
linkFont.size = '4'; linkFont.color = '#000000';
linkFont.append(...link.childNodes); link.append(linkFont);
}
// 只用于编辑器图片适配;帖子展示仍交给网站原有图片/附件规则。
for (const img of articleFont.querySelectorAll('img')) {
img.style.maxWidth = '100%'; img.style.height = 'auto';
}
const source = doc.createElement('p');
source.dataset.dww11WxSource = '1';
const font = doc.createElement('font');
font.size = '1'; font.color = '#c0c0c0';
const lines = (sourceText ?? `来源:${author},如有侵权,请联系删除。`).split('\n');
lines.forEach((line, index) => {
if (index) font.append(doc.createElement('br'));
font.append(doc.createTextNode(line));
});
source.append(font); body.append(source);
}
function existingSource(body) {
// 按段落和 BR 读取末尾逻辑行,兼容嵌套 FONT、纯文本及末尾空段落。
const lines = [];
let parts = [];
function flush() {
const text = cleanText(parts.map(part => part.node.textContent.slice(part.start, part.end)).join(''));
if (text) lines.push({ text, parts });
parts = [];
}
function visit(node) {
if (node.nodeType === 3) {
// 保留文本中的显式换行,并记录精确位置,避免删除同容器中的正文。
const value = node.textContent;
let start = 0;
for (let i = 0; i <= value.length; i++) {
if (i !== value.length && value[i] !== '\n') continue;
// HTML 缩进换行通常只是空白;只有后续是来源或版权行才分行。
if (i < value.length && !/^\s*(?:(?:来源|文章来源|本文来源|图文来源|文字来源|作者|作者信息|来源作者|转自|转载自|出处)\s*[::]|如有侵权|若有侵权|侵权请|转载请|版权归)/.test(value.slice(i + 1))) continue;
if (i > start) parts.push({ node, start, end: i });
if (i < value.length) flush();
start = i + 1;
}
return;
}
if (node.nodeType !== 1 || ['SCRIPT', 'STYLE', 'NOSCRIPT'].includes(node.tagName)) return;
if (node.tagName === 'BR') { flush(); return; }
if (['IMG', 'VIDEO', 'AUDIO', 'IFRAME', 'HR'].includes(node.tagName)) {
flush(); lines.push({ text: '\ufffc', parts: [] }); return;
}
const block = /^(P|DIV|SECTION|ARTICLE|FIGURE|FIGCAPTION|H[1-6]|LI|TR|BLOCKQUOTE|PRE)$/.test(node.tagName);
if (block) flush();
for (const child of node.childNodes) visit(child);
if (block) flush();
}
for (const child of body.childNodes) visit(child);
flush();
let first = -1;
for (let i = lines.length - 1; i >= 0; i--) {
if (/^(?:来源|文章来源|本文来源|图文来源|文字来源|作者|作者信息|来源作者|转自|转载自|出处)\s*[::]\s*\S/.test(lines[i].text)) {
first = i;
} else if (!/^(?:如有侵权|若有侵权|侵权请|转载请|版权归)/.test(lines[i].text)) {
break;
}
}
if (first < 0) return null;
const start = lines[first].parts[0];
const range = body.ownerDocument.createRange();
range.setStart(start.node, start.start);
range.setEnd(body, body.childNodes.length);
return {
text: lines.slice(first).map(line => line.text).join('\n'),
remove: () => range.deleteContents()
};
}
async function runFormat(button) {
if (busy) return;
setBusy(true);
try {
await ensureVisualEditor();
const body = editorBody();
const source = existingSource(body);
const hasText = cleanText(body.textContent).replace(source ? cleanText(source.text) : '', '').trim();
if (!hasText && !body.querySelector('img,video,audio,iframe,hr,table')) {
throw new Error('正文为空,请先输入或粘贴文章。');
}
const raw = source ? '' : prompt('未在文末检测到来源/作者信息,请补充来源公众号名称(取消则停止排版):');
if (raw === null) return;
const author = cleanText(raw);
if (!source && !author) throw new Error('来源公众号名称不能为空。');
button.textContent = '正在排版…';
// 已有来源保留原文,仅调整字号颜色;没有时才生成来源行。
source?.remove();
formatBody(body, author, source?.text ?? null);
alert(`排版完成。\n正文:4号黑色;来源:1号银色。\n${source ? source.text : `来源:${author}`}\n请人工检查后自行决定是否保存或发布。`);
} catch (error) { alert('排版未完成:' + error.message); }
finally {
setBusy(false);
button.textContent = '一键排版';
}
}
async function run(button) {
if (busy) return;
const raw = prompt('粘贴微信公众号文章 URL:');
if (raw === null || !raw.trim()) return;
setBusy(true);
let watcher;
try {
const url = normalizeURL(raw);
const jq = unsafeWindow.myJQ;
if (typeof jq !== 'function' || !jq.fn?.on) throw new Error('没有找到网站微信采集器的 myJQ,请刷新页面后重试。');
const plugin = document.querySelector('#e_controls .wxcj');
if (!plugin) throw new Error('当前页面没有微信采集组件。');
button.textContent = '读取标题和作者…';
const info = await getMetadata(url);
if (!info) return;
const subject = document.querySelector('#subject');
if (!subject) throw new Error('没有找到标题框。');
if (subject.maxLength > 0 && info.title.length > subject.maxLength) throw new Error('完整标题超过标题框长度限制,请先处理标题长度。');
await ensureVisualEditor();
button.textContent = '采集文章中…';
plugin.click();
const input = await waitFor(() => Array.from(document.querySelectorAll('.wxjc-dialog input')).find(visible));
const submit = input.closest('.wxjc-dialog').querySelector('.wxjc-subimt');
if (!submit) throw new Error('没有找到采集按钮。');
input.value = url;
input.dispatchEvent(new Event('input', { bubbles: true }));
input.dispatchEvent(new Event('change', { bubbles: true }));
watcher = watchImport(jq, url);
// 先登记失败处理,避免点击失败时留下未处理的 Promise。
watcher.result.catch(() => {});
submit.click();
await watcher.result;
button.textContent = '正在排版…';
const body = await waitFor(() => editorBody()?.innerHTML.trim() && editorBody());
formatBody(body, info.author);
subject.value = info.title;
subject.dispatchEvent(new Event('input', { bubbles: true }));
subject.dispatchEvent(new Event('change', { bubbles: true }));
subject.dispatchEvent(new KeyboardEvent('keyup', { bubbles: true }));
// 正文保持在可视化编辑器,由网站正常提交流程转成 BBCode。
// 不把 HTML 写入纯文本 message,也不调用任何保存/发布入口。
alert(`导入并排版完成。\n标题:${info.title}\n作者:${info.author}\n正文:4号黑色;来源:1号银色。\n请人工检查后自行决定是否保存或发布。`);
} catch (error) { alert('导入未完成:' + error.message); }
finally {
watcher?.cleanup();
setBusy(false);
button.textContent = '一键导入微信公众号并排版';
}
}
function mount() {
const toolbar = document.querySelector('#e_controls');
if (!toolbar || !document.querySelector('#subject')) return;
const style = 'position:relative;z-index:2;padding:0 12px;height:32px;line-height:30px;border:1px solid #159447;border-radius:4px;background:#159447;color:white;font-size:13px;cursor:pointer;';
// 右浮动的同一组内使用 flex,保证排版按钮始终位于导入按钮左侧。
let group = document.getElementById('dww11-wx-actions');
if (!group) {
group = document.createElement('span');
group.id = 'dww11-wx-actions';
group.style.cssText = 'float:right;display:inline-flex;align-items:center;gap:8px;margin:6px 8px;';
toolbar.append(group);
}
if (!document.getElementById(FORMAT_ID)) {
const button = document.createElement('button');
button.id = FORMAT_ID; button.type = 'button';
button.textContent = '一键排版'; button.style.cssText = style;
button.disabled = busy;
button.addEventListener('click', () => runFormat(button));
group.prepend(button);
}
if (toolbar.querySelector('.wxcj') && !document.getElementById(ID)) {
const button = document.createElement('button');
button.id = ID; button.type = 'button';
button.textContent = '一键导入微信公众号并排版'; button.style.cssText = style;
button.disabled = busy;
button.addEventListener('click', () => run(button));
group.append(button);
}
}
mount();
const observer = new MutationObserver(mount);
observer.observe(document.documentElement, { childList: true, subtree: true });
})();
+195
View File
@@ -0,0 +1,195 @@
// Extracted from dww11-wechat-oneclick.user.js v1.0.9; package source is authoritative.
(() => {
'use strict';
const cleanText = value => String(value || '').replace(/\s+/g, ' ').trim();
function flattenParagraphs(body) {
const selector = 'p,div,section,article,figure,figcaption,h1,h2,h3,h4,h5,h6';
const marker = () => {
const br = body.ownerDocument.createElement('br');
br.dataset.dww11Boundary = '1'; return br;
};
for (const block of Array.from(body.querySelectorAll(selector)).reverse()) {
if (block.closest('table,ul,ol,blockquote')) continue;
block.replaceWith(marker(), ...block.childNodes, marker());
}
let previous = false, pending = [];
function finish() {
const actual = pending.filter(br => !br.hasAttribute('data-dww11-boundary'));
let retained = false;
for (const br of pending) {
if (!br.hasAttribute('data-dww11-boundary')) continue;
if (previous && !actual.length && !retained) { br.removeAttribute('data-dww11-boundary'); retained = true; }
else br.remove();
}
pending = [];
}
const walker = body.ownerDocument.createTreeWalker(body, NodeFilter.SHOW_ELEMENT | NodeFilter.SHOW_TEXT);
const nodes = []; while (walker.nextNode()) nodes.push(walker.currentNode);
for (const node of nodes) {
if (node.nodeType === 1 && node.tagName === 'BR') pending.push(node);
else if ((node.nodeType === 3 && node.textContent.trim()) || (node.nodeType === 1 && ['IMG', 'VIDEO', 'HR', 'IFRAME'].includes(node.tagName))) {
finish(); previous = true;
}
}
// 末尾不保留由容器边界制造的额外换行。
for (const br of pending) if (br.hasAttribute('data-dww11-boundary')) br.remove();
}
function spaceImages(body) {
let previous = null, gapNodes = [], blankLines = 0;
const changes = [];
function content(kind, node) {
if (previous && previous.kind !== kind) {
let anchor = node;
while (anchor.parentElement && anchor.parentElement !== body &&
!['TD', 'TH', 'LI', 'BLOCKQUOTE'].includes(anchor.parentElement.tagName) && !anchor.parentElement.contains(previous.node)) anchor = anchor.parentElement;
changes.push({ anchor, nodes: gapNodes.slice(), count: Math.max(2, blankLines) });
}
previous = { kind, node }; gapNodes = []; blankLines = 0;
}
function visit(node) {
if (node.nodeType === 3) {
if (node.textContent.trim()) content('text', node); else gapNodes.push(node);
return;
}
if (node.nodeType !== 1 || ['SCRIPT', 'STYLE', 'NOSCRIPT'].includes(node.tagName)) return;
if (node.tagName === 'BR') { gapNodes.push(node); blankLines++; return; }
if (node.tagName === 'IMG') { content('image', node); return; }
for (const child of node.childNodes) visit(child);
}
for (const child of body.childNodes) visit(child);
for (const { anchor, nodes, count } of changes) {
for (const node of nodes) node.remove();
const fragment = body.ownerDocument.createDocumentFragment();
for (let i = 0; i < count; i++) fragment.append(body.ownerDocument.createElement('br'));
anchor.parentNode.insertBefore(fragment, anchor);
}
}
function formatBody(body, author, sourceText = null) {
const doc = body.ownerDocument;
body.querySelectorAll('[data-dww11-wx-source="1"]').forEach(el => el.remove());
// Discuz 会把内联 color 值直接转为 [color=…]。
// !important 不能出现在该值中;颜色和字号只用原生 FONT 属性。
for (const el of [body, ...body.querySelectorAll('*')]) {
if (['SCRIPT', 'STYLE'].includes(el.tagName)) continue;
for (const property of ['font', 'font-size', 'color', '-webkit-text-fill-color']) {
el.style.removeProperty(property);
}
// 其他排版属性也不保留 CSS 优先级,避免它泄漏进论坛标签。
for (const property of Array.from(el.style)) {
if (el.style.getPropertyPriority(property)) {
el.style.setProperty(property, el.style.getPropertyValue(property));
}
}
if (!el.style.cssText) el.removeAttribute('style');
}
// 去掉采集器或旧版脚本生成的嵌套 FONT,保留其中的内容和图片。
for (const font of Array.from(body.querySelectorAll('font'))) {
font.replaceWith(...font.childNodes);
}
flattenParagraphs(body);
spaceImages(body);
// 所有正文内容和换行放在同一 FONT 中。不能使用只有 BR 的空 FONT:
// 论坛转换器可能清除空格式标签,导致空行退回默认1号或直接丢失。
const articleFont = doc.createElement('font');
articleFont.size = '4'; articleFont.color = '#000000';
articleFont.dataset.dww11WxBody = '1';
articleFont.append(...body.childNodes);
body.append(articleFont);
// A 自身的浏览器默认蓝色会覆盖外层继承色,链接文字另设黑色。
for (const link of articleFont.querySelectorAll('a')) {
const linkFont = doc.createElement('font');
linkFont.size = '4'; linkFont.color = '#000000';
linkFont.append(...link.childNodes); link.append(linkFont);
}
// 只用于编辑器图片适配;帖子展示仍交给网站原有图片/附件规则。
for (const img of articleFont.querySelectorAll('img')) {
img.style.maxWidth = '100%'; img.style.height = 'auto';
}
const source = doc.createElement('p');
source.dataset.dww11WxSource = '1';
const font = doc.createElement('font');
font.size = '1'; font.color = '#c0c0c0';
const lines = (sourceText ?? `来源:${author},如有侵权,请联系删除。`).split('\n');
lines.forEach((line, index) => {
if (index) font.append(doc.createElement('br'));
font.append(doc.createTextNode(line));
});
source.append(font); body.append(source);
}
function existingSource(body) {
// 按段落和 BR 读取末尾逻辑行,兼容嵌套 FONT、纯文本及末尾空段落。
const lines = [];
let parts = [];
function flush() {
const text = cleanText(parts.map(part => part.node.textContent.slice(part.start, part.end)).join(''));
if (text) lines.push({ text, parts });
parts = [];
}
function visit(node) {
if (node.nodeType === 3) {
// 保留文本中的显式换行,并记录精确位置,避免删除同容器中的正文。
const value = node.textContent;
let start = 0;
for (let i = 0; i <= value.length; i++) {
if (i !== value.length && value[i] !== '\n') continue;
// HTML 缩进换行通常只是空白;只有后续是来源或版权行才分行。
if (i < value.length && !/^\s*(?:(?:来源|文章来源|本文来源|图文来源|文字来源|作者|作者信息|来源作者|转自|转载自|出处)\s*[::]|如有侵权|若有侵权|侵权请|转载请|版权归)/.test(value.slice(i + 1))) continue;
if (i > start) parts.push({ node, start, end: i });
if (i < value.length) flush();
start = i + 1;
}
return;
}
if (node.nodeType !== 1 || ['SCRIPT', 'STYLE', 'NOSCRIPT'].includes(node.tagName)) return;
if (node.tagName === 'BR') { flush(); return; }
if (['IMG', 'VIDEO', 'AUDIO', 'IFRAME', 'HR'].includes(node.tagName)) {
flush(); lines.push({ text: '\ufffc', parts: [] }); return;
}
const block = /^(P|DIV|SECTION|ARTICLE|FIGURE|FIGCAPTION|H[1-6]|LI|TR|BLOCKQUOTE|PRE)$/.test(node.tagName);
if (block) flush();
for (const child of node.childNodes) visit(child);
if (block) flush();
}
for (const child of body.childNodes) visit(child);
flush();
let first = -1;
for (let i = lines.length - 1; i >= 0; i--) {
if (/^(?:来源|文章来源|本文来源|图文来源|文字来源|作者|作者信息|来源作者|转自|转载自|出处)\s*[::]\s*\S/.test(lines[i].text)) {
first = i;
} else if (!/^(?:如有侵权|若有侵权|侵权请|转载请|版权归)/.test(lines[i].text)) {
break;
}
}
if (first < 0) return null;
const start = lines[first].parts[0];
const range = body.ownerDocument.createRange();
range.setStart(start.node, start.start);
range.setEnd(body, body.childNodes.length);
return {
text: lines.slice(first).map(line => line.text).join('\n'),
remove: () => range.deleteContents()
};
}
window.DWW11Formatter = Object.freeze({
format(expectedSource) {
if (Number(window.wysiwyg) !== 1) throw new Error('请切换到可视化编辑器后排版');
const body = document.querySelector('#e_iframe')?.contentDocument?.body;
if (!body) throw new Error('编辑器未就绪');
const footer = existingSource(body);
const expected = `来源:${expectedSource},如有侵权,请联系删除。`;
if (!footer || footer.text !== expected) throw new Error('文末来源与文章包不一致,请人工检查');
const images = [...body.querySelectorAll('img')].map(x => x.id + ':' + x.getAttribute('src'));
const backup = body.innerHTML;
try {
footer.remove();
formatBody(body, expectedSource, expected);
const after = [...body.querySelectorAll('img')].map(x => x.id + ':' + x.getAttribute('src'));
if (JSON.stringify(images) !== JSON.stringify(after)) throw new Error('排版后图片标识或顺序改变');
if (body.querySelectorAll('[data-dww11-wx-source="1"]').length !== 1) throw new Error('来源尾注数量异常');
body.dispatchEvent(new Event('input', {bubbles:true}));
return {images: after, source: expected};
} catch (error) { body.innerHTML = backup; throw error; }
}
});
})();
+480
View File
@@ -0,0 +1,480 @@
// ==UserScript==
// @name 广谈大湾网 · 本地文章目录导入
// @namespace local.dww11.article-importer
// @version 1.0.0
// @description 选择整个文章目录,导入 Markdown 标题、正文与本地配图;保留人工发布与程序接口。
// @match https://www.dww11.com/forum.php*
// @match https://dww11.com/forum.php*
// @run-at document-idle
// @grant none
// @noframes
// @license MIT
// ==/UserScript==
(function (root, factory) {
'use strict';
const core = factory();
if (typeof module === 'object' && module.exports) module.exports = core;
else core.mount(root);
})(typeof window === 'undefined' ? null : window, function () {
'use strict';
const VERSION = '1.0.0';
const MAX_IMAGE_SIZE = 20 * 1024 * 1024;
const IMAGE_EXT = /\.(?:jpe?g|png|gif)$/i;
function normalizePath(path) {
const parts = [];
const value = String(path).replace(/\\/g, '/').normalize('NFC');
if (/^(?:\/|[a-z]:|[a-z]+:)/i.test(value)) throw new Error('请使用目录内的相对路径:' + path);
for (const part of value.split('/')) {
if (!part || part === '.') continue;
if (part === '..') {
if (!parts.length) throw new Error('图片路径超出所选目录:' + path);
parts.pop();
} else parts.push(part);
}
return parts.join('/');
}
function imagePath(mdPath, target) {
if (/^(?:[a-z][\w+.-]*:|\/\/|\/)/i.test(target)) throw new Error('图片必须在所选目录内,请先下载:' + target);
let decoded;
try { decoded = decodeURIComponent(target); } catch { throw new Error('图片路径编码无效:' + target); }
return normalizePath(mdPath.slice(0, mdPath.lastIndexOf('/') + 1) + decoded);
}
// A small, intentionally bounded Markdown reader. Raw HTML is never executed.
// Links/images support balanced parentheses, angle destinations and optional titles.
function readLink(text, start) {
const image = text.startsWith('![', start);
if (!image && text[start] !== '[') return null;
const left = start + (image ? 2 : 1);
let end = left;
for (; end < text.length; end++) {
if (text[end] === '\\') { end++; continue; }
if (text[end] === ']') break;
}
if (text.slice(end, end + 2) !== '](') return null;
let pos = end + 2, depth = 1, quoted = '', angle = false;
for (; pos < text.length; pos++) {
const ch = text[pos];
if (ch === '\\') { pos++; continue; }
if (quoted) { if (ch === quoted) quoted = ''; continue; }
if (!angle && depth === 1 && /\s/.test(text[pos - 1]) && (ch === '"' || ch === "'")) { quoted = ch; continue; }
if (ch === '<') { angle = true; continue; }
if (ch === '>') { angle = false; continue; }
if (!angle && ch === '(') depth++;
if (!angle && ch === ')' && --depth === 0) break;
}
if (depth) return null;
let target = text.slice(end + 2, pos).trim();
const destination = target.match(/^<([^>]+)>\s*(?:["'][\s\S]*["'])?$/);
target = destination ? destination[1] : target.replace(/\s+["'][\s\S]*["']\s*$/, '');
target = target.replace(/\\([()\\ ])/g, '$1');
return { image, label: text.slice(left, end), target, end: pos + 1 };
}
function inline(text, onImage) {
let out = '';
for (let pos = 0; pos < text.length;) {
if (text[pos] === '\\' && pos + 1 < text.length) { out += text[pos + 1]; pos += 2; continue; }
const link = readLink(text, pos);
if (link) {
if (link.image) out += onImage(link.target, link.label);
else if (/^(https?:\/\/|mailto:)/i.test(link.target) && !/[\[\]\r\n"']/.test(link.target)) {
out += '[url=' + link.target + ']' + inline(link.label, onImage) + '[/url]';
} else out += link.label + ' (' + link.target + ')';
pos = link.end;
continue;
}
if (text[pos] === '`') {
const end = text.indexOf('`', pos + 1);
if (end > pos + 1) { out += text.slice(pos + 1, end); pos = end + 1; continue; }
}
let matched = false;
for (const [mark, tag] of [['**', 'b'], ['__', 'b'], ['~~', 's'], ['*', 'i'], ['_', 'i']]) {
if (!text.startsWith(mark, pos) || (mark === '_' && /\w/.test(text[pos - 1] || ''))) continue;
const end = text.indexOf(mark, pos + mark.length);
if (end <= pos + mark.length) continue;
out += '[' + tag + ']' + inline(text.slice(pos + mark.length, end), onImage) + '[/' + tag + ']';
pos = end + mark.length; matched = true; break;
}
if (!matched) out += text[pos++];
}
return out;
}
function markdownToBBCode(markdown, onImage) {
const lines = markdown.replace(/\r\n?/g, '\n').split('\n');
const output = [];
let list = '', quote = false, code = false, fence = '';
const closeList = () => { if (list) output.push('[/list]'); list = ''; };
for (let index = 0; index < lines.length; index++) {
const line = lines[index];
const codeMark = line.match(/^\s*(`{3,}|~{3,})/);
if (codeMark && (!code || codeMark[1][0] === fence)) {
closeList();
output.push(code ? '[/code]' : '[code]');
code = !code; fence = codeMark[1][0]; continue;
}
if (code) { output.push(line.replace(/\[\/code\]/gi, '[/code]')); continue; }
const quoted = /^\s*>\s?/.test(line);
if (quoted !== quote) { closeList(); output.push(quoted ? '[quote]' : '[/quote]'); quote = quoted; }
const content = quoted ? line.replace(/^\s*>\s?/, '') : line;
const item = content.match(/^\s*(?:([-+*])|\d+[.)])\s+(.+)$/);
if (item) {
const type = item[1] ? '[list]' : '[list=1]';
if (list !== type) { closeList(); output.push(type); list = type; }
output.push('[*]' + inline(item[2], onImage)); continue;
}
closeList();
const heading = content.match(/^\s{0,3}(#{1,6})\s+(.+?)\s*#*$/);
if (heading) output.push('[b]' + inline(heading[2], onImage) + '[/b]');
else if (/^\s*(?:---+|\*\*\*+|___+)\s*$/.test(content)) output.push('[hr]');
else output.push(inline(content, onImage));
}
closeList();
if (quote) output.push('[/quote]');
if (code) output.push('[/code]');
return output.join('\n').trim();
}
function compile(markdown, mdPath, fileMap, titleOverride) {
let body = String(markdown).replace(/^\uFEFF/, '').replace(/\r\n?/g, '\n');
const firstHeading = body.match(/^\s*#\s+([^\n]+)\n?/);
const title = String(titleOverride || firstHeading?.[1] || mdPath.split('/').pop().replace(/\.md$/i, '')).trim();
if (firstHeading) body = body.slice(firstHeading[0].length).trim();
if (!title || Array.from(title).length > 80) throw new Error('标题为空或超过 80 字,请修改 Markdown 标题后重试。');
if (!body.trim()) throw new Error('正文为空。');
const warnings = [];
if (/<\/?[a-z][^>]*>/i.test(body)) warnings.push('存在 HTML,将作为文字保留;请检查排版。');
if (/^\s*\|.*\|\s*$/m.test(body)) warnings.push('表格暂按原文保留,需手动检查。');
if (/!\[[^\]]*\]\s*\[[^\]]*\]/.test(body) || /^\s*\[[^\]]+\]:/m.test(body)) {
throw new Error('暂不支持引用式 Markdown 图片/链接,请改成 ![](图片路径) 或 [文字](网址)。');
}
const images = new Map();
// Validate every image before any network request; repeated references upload once.
markdownToBBCode(body, (target) => {
const path = imagePath(mdPath, target);
const file = fileMap.get(path);
if (!file) throw new Error('找不到配图:' + path);
if (!IMAGE_EXT.test(path)) throw new Error('网站仅支持 JPG、JPEG、PNG、GIF:' + path);
if (!file.size || file.size >= MAX_IMAGE_SIZE) throw new Error('配图为空或达到 20 MB:' + path);
images.set(path, file);
return '[图片:' + path + ']';
});
return { title, body, mdPath, images, warnings,
render(aids) {
return markdownToBBCode(body, target => {
const path = imagePath(mdPath, target);
const aid = aids.get(path);
if (!/^[1-9]\d*$/.test(String(aid))) throw new Error('配图尚未上传:' + path);
return '[attachimg]' + aid + '[/attachimg]';
});
}
};
}
async function prepareFiles(files, markdownPath) {
const fileMap = new Map();
for (const file of Array.from(files)) {
const path = normalizePath(file.webkitRelativePath || file.name);
if (fileMap.has(path)) throw new Error('文件路径重复:' + path);
fileMap.set(path, file);
}
const candidates = Array.from(fileMap.keys()).filter(p => /\.md$/i.test(p)).sort();
if (!candidates.length) throw new Error('所选目录中没有 .md 文件。');
if (!markdownPath && candidates.length > 1) return { candidates, files: Array.from(files) };
const mdPath = markdownPath || candidates[0];
const md = fileMap.get(mdPath);
if (!md) throw new Error('指定的 Markdown 文件不存在。');
return compile(await md.text(), mdPath, fileMap);
}
async function preparePackage(data) {
if (!data || data.schemaVersion !== 1 || typeof data.markdown !== 'string' || !Array.isArray(data.images)) {
throw new Error('文章包格式错误,需要 schemaVersion: 1、markdown 和 images。');
}
const map = new Map();
for (const item of data.images) {
const path = normalizePath(item.path);
if (map.has(path)) throw new Error('文章包图片路径重复:' + path);
let blob = item.blob;
if (!blob && typeof item.base64 === 'string') {
const binary = atob(item.base64);
blob = new Blob([Uint8Array.from(binary, ch => ch.charCodeAt(0))], { type: item.mime || 'application/octet-stream' });
}
if (!blob || typeof blob.arrayBuffer !== 'function') throw new Error('文章包缺少图片数据:' + path);
map.set(path, blob);
}
return compile(data.markdown, normalizePath(data.markdownPath || 'article.md'), map, data.title);
}
function parseAid(text) {
const value = String(text).trim();
// The site's actual WebUploader adapter uses a positive integer response.
if (!/^(?:[1-9]\d*|"[1-9]\d*")$/.test(value)) throw new Error('服务器未返回有效附件编号(可能登录过期、超出权限或上传失败)。');
return value.replace(/"/g, '');
}
function mount(win) {
const doc = win.document;
const params = new URLSearchParams(win.location.search);
if (params.get('mod') !== 'post' || params.get('action') !== 'newthread' || !doc.querySelector('#subject') || !doc.querySelector('#e_textarea')) return;
if (win.DWW11Importer) return;
let article = null, busy = false, candidates = null, lastResult = null, savedBackup = null, stageRevision = 0;
const cache = new Map(), urls = [], logs = [];
const host = doc.createElement('div');
host.id = 'dww11-importer';
host.style.cssText = 'position:fixed;right:18px;bottom:18px;z-index:2147483000';
const shadow = host.attachShadow({ mode: 'open' });
shadow.innerHTML = `<style>
:host{all:initial}*{box-sizing:border-box}details{font:14px/1.6 system-ui,sans-serif;color:#203044;background:#fff;border:1px solid #d4dee8;border-radius:12px;box-shadow:0 6px 28px #0003;width:360px;max-width:92vw}summary{cursor:pointer;padding:12px 16px;font-weight:650}section{padding:0 16px 16px;max-height:70vh;overflow:auto}button,select{font:inherit;padding:7px 10px;border-radius:6px;border:1px solid #b8c9da;background:#f4f8fc;color:#203044;cursor:pointer;margin:4px 4px 4px 0}button.primary{background:#1767b3;color:white;border-color:#1767b3}button:disabled{opacity:.5;cursor:default}p{margin:8px 0}small{color:#617183}pre{font:12px/1.6 system-ui;white-space:pre-wrap;overflow-wrap:anywhere;max-height:170px;overflow:auto;background:#f4f6f9;padding:8px}img{max-width:90px;max-height:80px;margin:4px;object-fit:contain}.error{color:#b32424}.success{color:#18713e}select{max-width:100%}[hidden]{display:none!important}
</style><details open><summary>本地文章导入 · 1.0</summary><section>
<small>选择整个文章文件夹,检查后上传配图并填写。发布由你操作。网站自带的自动保存仍可能运行。</small>
<p><button id="choose" type="button">选择文章目录</button><button id="package" type="button">导入文章包</button></p>
<input id="folder" type="file" webkitdirectory multiple hidden><input id="json" type="file" accept=".json,application/json" hidden>
<select id="md" hidden aria-label="选择 Markdown"></select>
<p id="status" role="status" aria-live="polite">尚未选择目录</p><div id="preview" hidden><strong id="title"></strong><pre id="body"></pre><div id="images"></div><p id="warnings"></p></div>
<button id="import" class="primary" type="button" disabled>上传配图并填写</button>
<button id="diagnostics" type="button">导出诊断</button><pre id="log" hidden></pre>
</section></details>`;
doc.body.appendChild(host);
const $ = id => shadow.getElementById(id);
const summary = () => ({ version: VERSION, busy, ready: !!article, title: article?.title || null,
markdownPath: article?.mdPath || null, imageCount: article?.images.size || 0,
warnings: article?.warnings || [], result: lastResult });
function notify(phase, message, extra = {}) {
$('status').textContent = message;
$('status').className = phase === 'error' ? 'error' : phase === 'complete' ? 'success' : '';
logs.push({ time: new Date().toISOString(), phase, message, ...extra });
$('log').hidden = false;
$('log').textContent = logs.slice(-8).map(l => l.message).join('\n');
win.dispatchEvent(new CustomEvent('dww11-importer:progress', { detail: { phase, message, ...extra } }));
}
function releasePreviews() { while (urls.length) URL.revokeObjectURL(urls.pop()); }
function clearStage() {
article = null; lastResult = null; candidates = null; releasePreviews();
$('preview').hidden = true; $('md').hidden = true; $('import').disabled = true;
}
function showArticle(prepared) {
article = prepared; lastResult = null; releasePreviews();
$('preview').hidden = false;
$('title').textContent = article.title;
$('body').textContent = article.body;
$('warnings').textContent = article.warnings.join(';');
$('images').replaceChildren();
for (const [path, blob] of article.images) {
const img = doc.createElement('img');
img.src = URL.createObjectURL(blob); urls.push(img.src); img.alt = path; img.title = path;
$('images').appendChild(img);
}
$('import').disabled = false;
notify('ready', `已检查:${article.images.size} 张图片。点击“上传配图并填写”开始导入。`);
return summary();
}
async function stageFiles(files, options = {}) {
if (busy) throw new Error('正在导入,请稍候。');
clearStage();
const revision = ++stageRevision;
const list = Array.from(files);
const prepared = await prepareFiles(list, options.markdownPath);
if (revision !== stageRevision) return { cancelled: true };
if (prepared.candidates) {
candidates = prepared.files;
$('md').replaceChildren(new Option('请选择目录中的 Markdown 文件', ''));
for (const path of prepared.candidates) $('md').appendChild(new Option(path, path));
$('md').hidden = false;
notify('choose-markdown', '目录中有多个 Markdown 文件,请选择一篇。');
return { candidates: prepared.candidates };
}
return showArticle(prepared);
}
async function stagePackage(data) {
if (busy) throw new Error('正在导入,请稍候。');
clearStage();
const revision = ++stageRevision;
const prepared = await preparePackage(data);
if (revision !== stageRevision) return { cancelled: true };
return showArticle(prepared);
}
const snapshot = () => ({ title: doc.querySelector('#subject').value,
mode: Number(win.wysiwyg), text: win.getEditorContents() });
function assertEditor() {
for (const name of ['getEditorContents', 'switchEditor']) {
if (typeof win[name] !== 'function') throw new Error('未识别到网站编辑器:' + name + '。请刷新页面后重试。');
}
if (!win.imgUpload?.settings || typeof win.updateImageList !== 'function') throw new Error('网站图片上传组件尚未就绪,请刷新后重试。');
if (Number(win.allowbbcode) === 0 || Number(win.allowimgcode) === 0) throw new Error('当前版块或账号不支持正文格式/图片。');
}
async function uploadImage(blob, path, settings, signal) {
signal?.throwIfAborted();
const url = new URL(settings.upload_url, win.location.href);
if (url.origin !== win.location.origin || url.pathname !== '/misc.php' || url.searchParams.get('operation') !== 'upload') {
throw new Error('上传地址与预期不符,已停止。');
}
const hash = Array.from(new Uint8Array(await win.crypto.subtle.digest('SHA-256', await blob.arrayBuffer())), n => n.toString(16).padStart(2, '0')).join('');
const key = url.href + ':' + settings.post_params.uid + ':' + hash;
if (cache.has(key)) { notify('upload', '复用已上传图片:' + path); return cache.get(key); }
const data = new FormData();
for (const [key, value] of Object.entries(settings.post_params)) data.append(key, String(value));
data.set('type', 'image');
data.set('filetype', blob.type || (/\.png$/i.test(path) ? 'image/png' : /\.gif$/i.test(path) ? 'image/gif' : 'image/jpeg'));
data.append(settings.file_post_name || 'Filedata', blob, path.split('/').pop());
const control = new AbortController(), timer = setTimeout(() => control.abort(), 90000);
try {
signal?.throwIfAborted();
signal?.addEventListener('abort', () => control.abort(), { once: true });
const response = await win.fetch(url.href, { method: 'POST', body: data, credentials: 'same-origin', redirect: 'error', signal: control.signal });
if (!response.ok) throw new Error('图片上传请求失败:HTTP ' + response.status);
const aid = parseAid(await response.text());
cache.set(key, aid);
return aid;
} catch (error) {
if (error.name === 'AbortError') throw new Error('上传超时;该图片可能已到达服务器,请检查网站图片列表后再重试。');
throw error;
} finally { clearTimeout(timer); }
}
async function registerImages(aids, signal) {
signal?.throwIfAborted();
if (!aids.length) return;
const ids = new Set(aids);
doc.querySelectorAll('#imgattachlist [id^="image_td_"],#imgattachlist [id^="image_"]').forEach(el => {
const id = el.id.match(/^(?:image_td_|image_)(\d+)$/)?.[1]; if (id) ids.add(id);
});
const ready = () => aids.every(id => doc.getElementById('image_' + id));
if (ready()) return;
// Use the site's own list loader, which registers attachment form fields too.
win.updateImageList(1, Array.from(ids).join('|'));
await new Promise((resolve, reject) => {
const timer = setTimeout(() => { observer.disconnect(); reject(new Error('图片已上传,但网站图片列表加载超时。重试会复用本页已上传图片。')); }, 25000);
signal?.addEventListener('abort', () => { clearTimeout(timer); observer.disconnect(); reject(new Error('流程已终止')); }, { once: true });
const check = () => { if (ready()) { clearTimeout(timer); observer.disconnect(); resolve(); } };
const observer = new MutationObserver(check);
observer.observe(doc.body, { childList: true, subtree: true }); check();
});
}
function setEditor(title, bbcode) {
win.switchEditor(0);
if (Number(win.wysiwyg) !== 0) throw new Error('无法切换编辑器为纯文本模式。');
const textarea = doc.querySelector('#e_textarea');
textarea.value = bbcode;
doc.querySelector('#subject').value = title;
for (const el of [textarea, doc.querySelector('#subject')]) {
el.dispatchEvent(new Event('input', { bubbles: true })); el.dispatchEvent(new Event('change', { bubbles: true }));
}
// Let Discuz generate the WYSIWYG content with native attachment metadata.
win.switchEditor(1);
const switcher = doc.querySelector('#e_switchercheck');
if (switcher) switcher.checked = Number(win.wysiwyg) === 0;
}
async function importPrepared(options = {}) {
const { signal, automatic = false } = options;
signal?.throwIfAborted();
if (busy) throw new Error('正在导入,请勿重复点击。');
if (!article) throw new Error('请先选择文章目录或文章包。');
assertEditor();
for (const id of ['bbcodeoff', 'htmlon']) {
if (doc.getElementById(id)?.checked) throw new Error('请先关闭“禁用 Discuz! 代码”和“HTML 代码”,再导入。');
}
const before = snapshot();
if ((before.title.trim() || before.text.replace(/<[^>]*>/g, '').trim() || /<img\b/i.test(before.text)) &&
(automatic ? (() => { throw new Error('编辑器已有内容,请人工处理旧稿后重试'); })() : !win.confirm('当前编辑器已有内容。导入将替换标题和正文,是否继续?'))) return { cancelled: true };
const selected = article;
busy = true;
const controls = ['choose', 'package', 'import', 'md'].map($);
controls.forEach(el => { el.disabled = true; });
// Prevent accidental submission of a partly imported article.
const blockSubmit = event => { event.preventDefault(); event.stopImmediatePropagation(); notify('busy', '正在导入,请等待完成后再操作发布。'); };
const form = doc.querySelector('#postform');
form.addEventListener('submit', blockSubmit, true);
const submitButtons = Array.from(form.querySelectorAll('button[type="submit"],button[name],input[type="submit"]'));
const previousDisabled = submitButtons.map(el => el.disabled);
submitButtons.forEach(el => { el.disabled = true; });
let committed = false;
try {
const aids = new Map();
const settings = win.imgUpload.settings;
const max = Math.min(MAX_IMAGE_SIZE, Number(settings.file_size_limit) * 1024 || MAX_IMAGE_SIZE);
for (const [path, blob] of selected.images) if (blob.size >= max) throw new Error('配图超过当前网站限制:' + path);
let index = 0;
for (const [path, blob] of selected.images) {
notify('upload', `上传 ${++index}/${selected.images.size}:${path}`, { completed: index - 1, total: selected.images.size });
aids.set(path, await uploadImage(blob, path, settings, signal));
}
notify('register', '正在加载网站图片列表…');
await registerImages(Array.from(new Set(aids.values())), signal);
signal?.throwIfAborted();
const now = snapshot();
if ((now.title !== before.title || now.text !== before.text) && (automatic ? (() => { throw new Error('上传期间编辑器被修改,已保留当前内容'); })() : !win.confirm('上传期间你修改了编辑器。是否用导入文章替换最新内容?'))) {
notify('cancelled', '已保留当前正文;图片已上传,可稍后再次导入。'); return { cancelled: true };
}
const backup = snapshot();
const bbcode = selected.render(aids);
// Preserve a local, downloadable snapshot before changing title or body.
savedBackup = { title: backup.title, content: backup.text, mode: backup.mode, createdAt: new Date().toISOString() };
try {
for (const id of ['bbcodeoff', 'htmlon']) {
const el = doc.getElementById(id);
if (el && id === 'bbcodeoff' && el.checked) throw new Error('请先取消“禁用 Discuz! 代码”,再重试。');
if (el && id === 'htmlon' && el.checked) throw new Error('请先关闭“HTML 代码”,再重试。');
}
signal?.throwIfAborted();
setEditor(selected.title, bbcode);
const raw = win.getEditorContents();
if (Number(win.wysiwyg) === 1) {
for (const id of new Set(aids.values())) if (!raw.includes('attachimg_' + id)) throw new Error('富文本图片转换未通过检查。');
} else if (raw !== bbcode) throw new Error('正文填写校验失败。');
if (doc.querySelector('#subject').value !== selected.title) throw new Error('标题填写校验失败。');
committed = true;
} catch (error) {
// Restore the exact editor representation saved immediately before filling.
win.switchEditor(backup.mode);
if (backup.mode) win.editdoc.body.innerHTML = backup.text;
else doc.querySelector('#e_textarea').value = backup.text;
doc.querySelector('#subject').value = backup.title;
const switcher = doc.querySelector('#e_switchercheck'); if (switcher) switcher.checked = !backup.mode;
throw error;
}
lastResult = { title: selected.title, images: Object.fromEntries(aids), completedAt: new Date().toISOString() };
notify('complete', `已填写标题、正文和 ${aids.size} 张配图。请检查排版,随后手动发表。`);
return lastResult;
} catch (error) {
notify('error', error.message + (committed ? '' : ' 标题和正文未替换;已上传的图片会保留。'));
throw error;
} finally {
busy = false; form.removeEventListener('submit', blockSubmit, true);
submitButtons.forEach((el, index) => { el.disabled = previousDisabled[index]; });
controls.forEach(el => { el.disabled = false; });
$('import').disabled = !article;
}
}
function downloadJSON(value, filename) {
const url = URL.createObjectURL(new Blob([JSON.stringify(value, null, 2)], { type: 'application/json' }));
const link = doc.createElement('a'); link.href = url; link.download = filename; link.click();
setTimeout(() => URL.revokeObjectURL(url), 1000);
}
// Public v1 interface. Staging is read-only; importPrepared is the explicit upload/fill step.
// No postMessage listener, localhost service, publishing endpoint or saved credentials.
const api = Object.freeze({ version: VERSION, schemaVersion: 1, stageFiles, stagePackage,
getState: summary, openPicker: () => { if (!busy) $('folder').click(); }, importPrepared,
getBackup: () => savedBackup });
win.DWW11Importer = api;
$('choose').onclick = () => { $('folder').value = ''; $('folder').click(); };
$('package').onclick = () => { $('json').value = ''; $('json').click(); };
const handleError = error => notify('error', error.message || String(error));
$('folder').onchange = () => stageFiles($('folder').files).catch(handleError);
$('json').onchange = async () => {
try { const file = $('json').files[0]; if (file) await stagePackage(JSON.parse(await file.text())); }
catch (error) { handleError(error); }
};
$('md').onchange = () => {
const files = candidates, path = $('md').value;
if (files && path) stageFiles(files, { markdownPath: path }).catch(handleError);
};
$('import').onclick = () => importPrepared().catch(handleError);
$('diagnostics').onclick = () => downloadJSON({ ...summary(), logs }, 'dww11-import-diagnostics.json');
win.addEventListener('pagehide', releasePreviews, { once: true });
}
return { VERSION, normalizePath, imagePath, readLink, markdownToBBCode, compile, prepareFiles, preparePackage, parseAid, mount };
});
+73
View File
@@ -0,0 +1,73 @@
// ==UserScript==
// @name 新闻工作台 · 本地连接
// @namespace local.dww11.auto-news
// @version 0.1.0
// @match https://www.dww11.com/*
// @match https://dww11.com/*
// @match https://house.dww11.com/*
// @grant GM_xmlhttpRequest
// @grant GM_getValue
// @grant GM_setValue
// @grant GM_addValueChangeListener
// @connect 127.0.0.1
// @run-at document-start
// @noframes
// ==/UserScript==
(() => {
'use strict';
const allowed = new Set(['heartbeat','next','package','event','accept','switch-begin','takeover','stop','current']);
let token = GM_getValue('pairToken',''), port = GM_getValue('pairPort',8770);
let stopRevision = 0;
const signalStop = () => { stopRevision++; document.dispatchEvent(new CustomEvent('dww11:auto-stop')); };
GM_addValueChangeListener('halted', (_key, _old, value) => { if (value) signalStop(); });
function request(route, body) {
return new Promise((resolve, reject) => {
if (!token) return reject(new Error('请在左下角填入本地工作台的连接码'));
GM_xmlhttpRequest({method:'POST',url:`http://127.0.0.1:${port}/bridge/${route}`,
headers:{'Content-Type':'application/json','X-Session-Token':token},data:JSON.stringify(body),timeout:60000,
onload(response) { try { const value=JSON.parse(response.responseText); if(response.status!==200) throw new Error(value.error || '本地连接失败'); resolve(value); } catch(error) { reject(error); } },
onerror:()=>reject(new Error('本地服务未连接,请检查服务与脚本权限')),
ontimeout:()=>reject(new Error('本地服务超时,任务已暂停'))});
});
}
document.addEventListener('dww11:local-request', async event => {
let message;
try {message=JSON.parse(event.detail);} catch {return;}
if (!allowed.has(message.route) || typeof message.id !== 'string') return;
try {
if (message.route==='stop') { GM_setValue('halted',true); signalStop(); }
const revision = stopRevision;
let value=await request(message.route,message.body || {});
if (message.route==='heartbeat') {
if (revision !== stopRevision) value = {...value, running:false, reason:'终止后忽略旧连接响应'};
GM_setValue('halted',!value.running);
if (!value.running) signalStop();
}
document.dispatchEvent(new CustomEvent('dww11:local-response',{detail:JSON.stringify({id:message.id,value})}));
} catch(error) {
if (message.route==='heartbeat') {GM_setValue('halted',true); signalStop();}
document.dispatchEvent(new CustomEvent('dww11:local-response',{detail:JSON.stringify({id:message.id,error:error.message})}));
}
});
function mount() {
if (!document.body || document.getElementById('auto-news-connection')) return;
const host=document.createElement('div');host.id='auto-news-connection';
host.style.cssText='position:fixed;left:16px;bottom:16px;z-index:2147483647';
const shadow=host.attachShadow({mode:'closed'});
shadow.innerHTML='<style>section{font:13px/1.6 system-ui;background:white;color:#24364a;border:1px solid #ccd7e0;border-radius:10px;padding:12px;width:250px;box-shadow:0 3px 20px #0002}button,input{font:inherit;box-sizing:border-box;padding:7px;margin:4px 0;width:100%}button{cursor:pointer}#halt{background:#ad302d;color:white;border:0;border-radius:6px}small{display:block}</style><section><strong>新闻工作台 · 本地连接</strong><button id="halt">终止流程</button><details><summary>连接设置</summary><label>本地端口<input id="port" type="number" min="1" max="65535"></label><label>连接码<input id="token" type="password" autocomplete="off"></label><button id="save">保存连接设置</button></details><small id="status">请启动本地工作台并配对</small></section>';
const $=id=>shadow.getElementById(id);$('port').value=port;
$('save').onclick=()=>{
const value=Number($('port').value);if(!Number.isInteger(value)||value<1||value>65535){$('status').textContent='端口无效';return;}
if($('token').value.trim()) token=$('token').value.trim();port=value;
GM_setValue('pairToken',token);GM_setValue('pairPort',port);$('token').value='';$('status').textContent='已保存;服务重启后需要更新连接码';
};
$('halt').onclick=async()=>{
GM_setValue('halted',true);signalStop();$('status').textContent='已终止;在途请求仍可能完成';
try {await request('stop',{});} catch(error) {$('status').textContent+='。'+error.message;}
};
document.addEventListener('dww11:connection-status',event=>{$('status').textContent=String(event.detail).slice(0,250);});
document.body.append(host);
if(GM_getValue('halted',true)) signalStop();
}
if(document.readyState==='loading') document.addEventListener('DOMContentLoaded',mount,{once:true}); else mount();
})();
+173
View File
@@ -0,0 +1,173 @@
/* Page-side orchestrator. This file never clicks or submits the publication form. */
(() => {
'use strict';
if (window.DWW11NewsRunner) return;
window.DWW11NewsRunner = true;
const SLOT='auto-news-current';
const owner=sessionStorage.getItem('auto-news-owner') || crypto.randomUUID();
sessionStorage.setItem('auto-news-owner',owner);
let run=null, working=false, halted=true, aborter=null, active=null, tickBusy=false;
const pending=new Map();
function rpc(route,body={}) {
return new Promise((resolve,reject)=>{
const id=crypto.randomUUID();
const timer=setTimeout(()=>{pending.delete(id);reject(new Error('本地连接无响应'));},65000);
pending.set(id,{resolve,reject,timer});
document.dispatchEvent(new CustomEvent('dww11:local-request',{detail:JSON.stringify({id,route,body:{owner,...body}})}));
});
}
document.addEventListener('dww11:local-response',event=>{
let data;try{data=JSON.parse(event.detail);}catch{return;}
const waiter=pending.get(data.id);if(!waiter)return;
clearTimeout(waiter.timer);pending.delete(data.id);
data.error?waiter.reject(new Error(data.error)):waiter.resolve(data.value);
});
const status=message=>document.dispatchEvent(new CustomEvent('dww11:connection-status',{detail:message}));
document.addEventListener('dww11:auto-stop',()=>{halted=true;aborter?.abort(new Error('流程已终止'));});
function account() {
const element=document.querySelector('#mumucms_username');
if(!element)return null;
const name=[...element.childNodes].filter(n=>n.nodeType===Node.TEXT_NODE).map(n=>n.textContent).join('').trim();
const links=[element,...element.querySelectorAll('a')].filter(e=>e.href);
let uid=null;
for(const link of links) {
const href=link.href;
uid=new URL(href,location.href).searchParams.get('uid') || href.match(/space-uid-(\d+)/)?.[1];
if(uid)break;
}
return uid&&name?{uid,name}:null;
}
function extension() {
try{return JSON.parse(document.querySelector('#dww11-extension-panel')?.getAttribute('data-auto-news-state') || '{}');}catch{return {};}
}
function assertLive(epoch) {
if(halted||!run?.running||run.epoch!==epoch)throw new Error('流程已暂停或终止');
if(active?.account?.uid!==account()?.uid)throw new Error('当前账号与任务不一致');
}
function editorSnapshot() {
const body=document.querySelector('#e_iframe')?.contentDocument?.body;
if(!body)throw new Error('找不到可视化编辑器');
return {title:document.querySelector('#subject')?.value || '',
images:[...body.querySelectorAll('img')].map(x=>({id:x.id,src:x.getAttribute('src')})),
// Snapshot only final submitted content; never includes hidden tokens or the full form.
html:body.innerHTML.slice(0,500000), text:body.innerText.slice(0,100000)};
}
function saveLocal(value) {sessionStorage.setItem(SLOT,JSON.stringify(value));}
function loadLocal(){try{return JSON.parse(sessionStorage.getItem(SLOT)||'null');}catch{return null;}}
async function event(state,extra={}) {return rpc('event',{id:active.id,state,epoch:run.epoch,...extra});}
async function fail(error) {
status(error.message);
if(active)try{await event('needs_attention',{error:error.message});}catch{}
halted=true;aborter?.abort();document.dispatchEvent(new CustomEvent('dww11:auto-stop'));
}
function editorPage(job) {
const u=new URL(location.href);
return u.searchParams.get('mod')==='post'&&u.searchParams.get('action')==='newthread'&&u.searchParams.get('fid')===String(job.fid);
}
async function prepare(job) {
working=true;active=job;aborter=new AbortController();const epoch=run.epoch;
try {
assertLive(epoch);
if(!editorPage(job)) {
saveLocal({id:job.id});
location.assign(`https://www.dww11.com/forum.php?mod=post&action=newthread&fid=${job.fid}`);return;
}
if(!window.DWW11Importer||!window.DWW11Formatter)throw new Error('导入或排版模块尚未就绪,请检查安装');
status('正在检查文章包…');
const pack=await rpc('package',{id:job.id});assertLive(epoch);
const prepared=await window.DWW11Importer.stagePackage(pack);assertLive(epoch);
if(prepared.warnings?.length)throw new Error('文章需要人工处理:'+prepared.warnings.join(';'));
await event('importing');assertLive(epoch);status('上传图片并填写文章…');
const result=await window.DWW11Importer.importPrepared({automatic:true,signal:aborter.signal});assertLive(epoch);
if(result?.cancelled)throw new Error('导入已取消');
await event('formatting');assertLive(epoch);
window.DWW11Formatter.format(job.source);assertLive(epoch);
const snapshot=editorSnapshot();
if(snapshot.title!==job.title)throw new Error('标题校验失败');
const aids=new Set(Object.values(result.images));
for(const aid of aids) if(!snapshot.images.some(x=>x.id==='attachimg_'+aid))throw new Error('配图附件缺失:'+aid);
await event('awaiting_user_review',{snapshot});assertLive(epoch);
active={...job,state:'awaiting_user_review'};saveLocal({id:job.id});
status('已完成排版,请核对内容并亲手点击“发表帖子”。');
}catch(error){await fail(error);}finally{working=false;}
}
// Record only trusted user activation. Never replay a click, call submit(), or send a post request.
document.addEventListener('click',event=>{
if(!event.isTrusted||!event.target.closest?.('#postsubmit')||active?.state!=='awaiting_user_review')return;
try {
const snapshot=editorSnapshot();
const local={id:active.id,snapshot,clickedAt:Date.now(),observed:false};saveLocal(local);
rpc('event',{id:active.id,state:'user_submitting',epoch:run?.epoch,snapshot})
.then(()=>{const latest=loadLocal();if(latest?.clickedAt===local.clickedAt)saveLocal({...local,observed:true});})
.catch(error=>status('提交记录待恢复:'+error.message));
active.state='user_submitting';status('已观察到人工点击,等待结果页核对。');
}catch(error){status('提交快照记录失败:'+error.message);}
},true);
async function observeResult(job) {
const local=loadLocal();
if(local?.id===job.id&&local.snapshot&&job.state==='awaiting_user_review') {
await rpc('event',{id:job.id,state:'user_submitting',epoch:run.epoch,snapshot:local.snapshot});
job.state='user_submitting';
}
if(job.state==='user_submitting') {
// Result selectors intentionally await user-provided HTML; URL alone is not proof.
if(!editorPage(job) || (local?.clickedAt && Date.now()-local.clickedAt>45000)) {
await rpc('event',{id:job.id,state:'submit_unknown',error:'等待结果页 HTML 适配;候选页面:'+location.href});
await rpc('stop');
status('已记录候选结果页;尚不能自动核对。流程已暂停,请补充结果页 HTML。');
}
}else if(job.state==='submit_unknown')status('发布结果待核对;不会重发、计数或换号。');
}
async function switchAccount() {
if(working||halted)return;
if(location.origin!=='https://www.dww11.com'||location.pathname!=='/') {
location.assign('https://www.dww11.com/');return;
}
const ext=extension(),button=document.querySelector('#dww11-extension-next');
if(!ext.ready||!ext.idle||!button||button.disabled) {status('请安装并重新加载带联动接口的换号插件');return;}
await rpc('switch-begin');
if(halted||!run?.running)return;
button.click();status('正在换号;等待插件完成并核对新账号');
}
async function tick() {
if(tickBusy)return;tickBusy=true;
try {
run=await rpc('heartbeat',{account:account(),url:location.href,extension:extension()});
halted=!run.running;
if(!run.running) {aborter?.abort();document.dispatchEvent(new CustomEvent('dww11:auto-stop'));}
if(working)return;
// Passive result recording is permitted after Stop, but no new business actions.
if(run.active) {
const local=loadLocal();
if(local?.id===run.active&&local.snapshot&&!run.running)status('流程已暂停;提交结果仍需核对');
}
if(!run.running) {
if(run.active && run.owner===owner) {const saved=await rpc('current');if(saved.job) {active=saved.job;await observeResult(saved.job);}}
status(run.reason || '流程已暂停');return;
}
const next=await rpc('next',{account:account(),extension:extension()});run=next.run;
if(next.job)active=next.job;
if(next.action==='prepare'||(next.action==='resume'&&next.job.state==='preparing'))void prepare(next.job);
else if(next.action==='resume') {
await observeResult(next.job);
if(next.job.state==='awaiting_user_review') {
if(!editorPage(next.job)||!document.querySelector('#subject')?.value) await fail(new Error('待审稿页面已丢失,请人工核对旧稿,不自动重新导入'));
else status('等待你亲手发表;程序不再修改编辑器');
}
else if(['importing','formatting'].includes(next.job.state))await fail(new Error('导入阶段发生中断,请检查编辑器和已上传图片后处理任务'));
else if(next.job.state==='needs_attention')status(next.job.error);
}else if(next.action==='switch')await switchAccount();
else if(next.action==='switch_wait')status('等待换号结果;若插件已暂停,请人工处理后继续');
else if(next.action==='login')status('未能读取当前账号 UID,请登录并确认账号元素');
else if(next.action==='other_tab')status('另一个标签持有任务;如需接管请先暂停');
else if(next.action==='interval')status('等待设定的处理间隔');
else status(run.reason || '等待任务');
}catch(error){status(error.message);halted=true;aborter?.abort();document.dispatchEvent(new CustomEvent('dww11:auto-stop'));}
finally{tickBusy=false;}
}
const takeover=document.createElement('button');takeover.type='button';takeover.textContent='接管已暂停任务';
takeover.style.cssText='position:fixed;left:16px;top:16px;z-index:2147483646;padding:8px;background:#fff;border:1px solid #aab;border-radius:6px';
takeover.onclick=async()=>{try{await rpc('takeover');status('已接管;核对当前页面后在工作台继续');}catch(error){status(error.message);}};
document.body.append(takeover);
setInterval(tick,2000);tick();
})();
+23
View File
@@ -0,0 +1,23 @@
"""Build the page-context userscript from reviewed local components; no external assets."""
from pathlib import Path
ROOT = Path(__file__).resolve().parent
header = '''// ==UserScript==
// @name 新闻工作台 · 导入排版与流程
// @namespace local.dww11.auto-news
// @version 0.1.0
// @match https://www.dww11.com/*
// @match https://dww11.com/*
// @match https://house.dww11.com/*
// @grant none
// @run-at document-idle
// @noframes
// ==/UserScript==
'''
parts = []
for name in ('importer.js', 'formatter.js', 'runner.js'):
source = (ROOT / 'browser' / name).read_text()
if '// ==/UserScript==' in source:
source = source.split('// ==/UserScript==', 1)[1]
parts.append(source)
(ROOT / 'browser/auto-news.user.js').write_text(header + '\n'.join(parts))
print('已生成 browser/auto-news.user.js')
+57
View File
@@ -0,0 +1,57 @@
# 探数新闻下载器
在本机浏览器中填写 API Key、获取频道、选择数量,下载新闻正文和图片。所有项目文件与下载结果都位于本目录,不修改微信下载器。
## 启动
安装 `uv` 后,macOS 可以双击 `start.command`。也可以在本目录运行:
```sh
uv run --no-project --python 3.12 --with-requirements requirements.txt python app.py
```
启动后自动打开 http://127.0.0.1:8765 。保留启动窗口,退出时按 Ctrl+C。端口被占用时加 `--port 8766`;不自动打开浏览器时加 `--no-browser`。
1. 填写探数 API Key,点击「获取新闻频道」。频道来自真实接口,不使用写死的列表。
2. 选择频道、输入目标数量,点击「开始下载」。
3. 页面显示成功保存的数量、百分比、重复数、失败数和最近 300 条日志。日志固定高度,可滚动;向上查看时不会强制滚回底部。
4. 下载结果保存在本目录的 `downloads` 文件夹。点击「停止下载」后,等待当前请求或文章保存结束。
5. 「调试终端」默认折叠,支持频道查询以及列表查询的 `type`、`num`、`start` 参数。测试请求只显示响应,不保存文章;同样消耗 API 调用额度。输出最多保留 100000 字符。
密钥仅在页面和服务运行内存中使用,不保存到配置、历史数据库或浏览器存储。服务仅监听本机地址。刷新页面可查看当前任务进度,开始新任务需重新填写密钥。单个服务同时只运行一个下载任务,请勿同时启动多个实例操作同一下载目录。
## 保存格式
沿用 `wechat_official_acc_article_downloader` 的格式和图片处理规则:
```text
downloads/
├── .history.sqlite3
└── 新闻标题/
├── 新闻标题.md
├── image001.jpg
└── image002.png
```
Markdown 开头是一级标题,空行后直接接由接口 `content` 转换的正文,不额外添加接口 `time` 对应的发布时间或分隔线。末尾为 `来源:{src},如有侵权,请联系删除。`,来源为空时使用「未知作者」。不在开头添加原文链接或作者。正文中的图片下载到同一目录、使用相对引用、按真实文件类型命名,同一图片地址只下载一次。接口的 `pic` 封面不单独插入正文。
标题中的不合法字符会替换;不同文章碰巧同名时添加 ` (2)` 等后缀,目录和 Markdown 文件保持同名。任何正文图片下载失败,该篇不计入完成数,并清理半成品,再继续其他新闻;下次任务可以重试。
## 去重与分页
- 下载图片前,检查原文 URL / PC URL(移除常见跟踪参数和片段)及规范化正文指纹。任一匹配即跳过,标题相同但正文和网址不同不会单凭标题去重。
- SQLite 保存成功文章的指纹,跨任务、跨频道和服务重启后继续去重。被删除的文章目录不会阻止重新下载。不要删除 `.history.sqlite3`,否则会失去对应历史记录。
- 每次任务从最新位置 `start=0` 开始,按返回数量推进,每页最多 40 条;重复和失败不计入成功数量,并继续后面的新闻。
- 接口限制 `start ≤ 400`,一次任务最多覆盖约 440 条,具体还受返回数量和实时更新影响。达到分页边界或收到空列表时停止,未达目标就显示「未达到目标」,不会伪装成 100%。
- 列表接口不自动重试,防止隐蔽消耗计费额度。API 业务错误、网络错误或响应格式错误会中断任务,已成功保存的文章保留。图片请求沿用原下载器的重试策略。
- 新闻列表会实时变化,无法保证服务器在不同请求中不返回同一新闻;本工具在保存前去重,尽量避免重复图片下载与文件保存。
接口依据:[探数官方新闻 API 文档](https://www.tanshuapi.com/market/detail-85)。
## 验证
```sh
uv run --no-project --python 3.12 --with-requirements requirements.txt python -m unittest discover -s tests -v
```
测试使用模拟新闻响应与临时目录,不消耗真实 API 额度。覆盖格式与图片引用、图片失败清理、历史去重、分页边界、失败重试、取消、日志上限和本地 HTTP 接口。真实账号权限、余额、返回内容与远程图片可达性,需要在页面填入有效 Key 后验证。
+299
View File
@@ -0,0 +1,299 @@
#!/usr/bin/env python3
"""仅在本机运行的探数新闻下载器。"""
from contextlib import contextmanager
from collections import deque
from datetime import datetime
from hashlib import sha256
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import urlsplit, urlunsplit, parse_qsl, urlencode
import argparse
import json
import secrets
import sqlite3
import threading
import time
import unicodedata
import webbrowser
import requests
from bs4 import BeautifulSoup
from article_writer import save_article
ROOT = Path(__file__).resolve().parent
API = 'https://api.tanshuapi.com/api/toutiao/v1/'
def api_request(key, endpoint, params=None):
if endpoint not in ('channel', 'index'):
raise ValueError('未知接口')
try:
# 不自动重试计费接口,避免一次操作产生不可见的重复调用。
response = requests.get(API + endpoint, params={'key': key, **(params or {})}, timeout=(10, 40))
response.raise_for_status()
payload = response.json()
except requests.RequestException:
raise ValueError('接口网络请求失败或超时,请在调试终端检查后重试') from None
except ValueError:
raise ValueError('接口未返回有效 JSON') from None
if not isinstance(payload, dict):
raise ValueError('接口响应格式错误')
return payload
def result_list(payload):
if payload.get('code') != 1:
raise ValueError(f"探数接口错误 {payload.get('code')}:{payload.get('msg', '未知错误')}")
data = payload.get('data')
if not isinstance(data, dict) or not isinstance(data.get('list'), list):
raise ValueError('接口响应缺少 data.list')
return data['list']
def fingerprints(article):
values = []
for field in ('url', 'weburl'):
value = str(article.get(field) or '').strip()
if value:
u = urlsplit(value)
query = [(k, v) for k, v in parse_qsl(u.query) if not k.lower().startswith('utm_') and k.lower() not in ('spm', 'from', 'pos', 'vt')]
values.append('url:' + urlunsplit(('', u.netloc.lower(), u.path, urlencode(sorted(query)), '')))
body = BeautifulSoup(str(article.get('content') or ''), 'html.parser').get_text(' ', strip=True)
body = ''.join(unicodedata.normalize('NFKC', body).split())
if body:
values.append('body:' + body)
return [sha256(value.encode()).hexdigest() for value in values]
class Downloader:
def __init__(self, root=ROOT / 'downloads', requester=api_request, writer=save_article, on_saved=None):
self.root = Path(root).resolve()
self.root.mkdir(parents=True, exist_ok=True)
self.requester, self.writer = requester, writer
self.on_saved = on_saved
self.lock = threading.RLock()
self.stop = threading.Event()
self.thread = None
self.logs = deque(maxlen=300)
self.seq = 0
self.state = dict(status='idle', saved=0, target=0, skipped=0, failed=0, scanned=0, requests=0, channel='', message='等待开始')
with self.db() as db:
db.execute('CREATE TABLE IF NOT EXISTS seen (fingerprint TEXT PRIMARY KEY, directory TEXT NOT NULL)')
@contextmanager
def db(self):
connection = sqlite3.connect(self.root / '.history.sqlite3')
try:
with connection:
yield connection
finally:
connection.close()
def log(self, message):
with self.lock:
self.seq += 1
self.logs.append(dict(id=self.seq, time=datetime.now().strftime('%H:%M:%S'), message=message))
def update(self, **values):
with self.lock:
self.state.update(values)
def snapshot(self):
with self.lock:
return {**self.state, 'logs': list(self.logs), 'output': str(self.root)}
def start(self, key, channel, target):
with self.lock:
if self.thread and self.thread.is_alive():
raise ValueError('已有任务正在运行')
self.stop.clear()
self.logs.clear()
self.state = dict(status='running', saved=0, target=target, skipped=0, failed=0, scanned=0, requests=0, channel=channel, message='正在下载')
self.thread = threading.Thread(target=self.run, args=(key, channel, target), daemon=True)
self.thread.start()
def run(self, key, channel, target):
saved = skipped = failed = scanned = calls = 0
seen_in_run = set()
self.log(f'开始下载频道「{channel}」,目标 {target} 条;历史重复新闻将跳过')
try:
with self.db() as db:
offset = 0
while offset <= 400 and saved < target and not self.stop.is_set():
count = min(40, target - saved)
calls += 1
self.update(requests=calls)
self.log(f'请求第 {calls} 页:start={offset},num={count}')
items = result_list(self.requester(key, 'index', {'type': channel, 'num': count, 'start': offset}))
if not items:
self.log('接口返回空列表,已停止')
break
for article in items:
if saved >= target or self.stop.is_set():
break
scanned += 1
try:
if not isinstance(article, dict):
raise ValueError('新闻条目格式错误')
keys = fingerprints(article)
existing = any(k in seen_in_run for k in keys)
for k in keys:
row = db.execute('SELECT directory FROM seen WHERE fingerprint=?', (k,)).fetchone()
if row and Path(row[0]).is_dir():
existing = True
seen_in_run.update(keys)
if existing:
skipped += 1
self.log('跳过重复新闻:' + str(article.get('title', '无标题'))[:160])
continue
result = self.writer(article, self.root)
try:
db.executemany('INSERT OR REPLACE INTO seen VALUES (?, ?)', [(k, result.directory) for k in keys])
db.commit()
except Exception:
# 保存目录尚未登记成功,回滚本次产物,避免下次重复保存。
import shutil
shutil.rmtree(result.directory, ignore_errors=True)
raise
saved += 1
if self.on_saved:
try:
self.on_saved(article, result, keys)
except Exception as error:
self.log('文章已保存,入队需处理:' + str(error).replace(key, '[已隐藏]')[:300])
self.log(f'下载新闻 {saved}/{target} 条:{result.title}')
except Exception as exc:
failed += 1
self.log(f'保存失败:{str(exc).replace(key, "[已隐藏]")[:400]}')
finally:
self.update(saved=saved, skipped=skipped, failed=failed, scanned=scanned)
# 避免越过 start=400;已消费的条目决定下一起始位置。
offset += len(items)
if offset > 400:
self.log('已到达接口允许的分页范围')
if saved < target and offset <= 400:
self.stop.wait(0.3)
status = 'completed' if saved >= target else 'cancelled' if self.stop.is_set() else 'partial'
message = f'已保存 {saved}/{target} 条,跳过重复 {skipped} 条,失败 {failed} 条'
if status == 'partial':
message += ';当前可获取新闻不足,未达到目标数量'
if status == 'cancelled':
message += ';任务已停止'
self.update(status=status, message=message)
self.log(message)
except Exception as exc:
message = str(exc).replace(key, '[已隐藏]')[:500]
self.update(status='error', message=message)
self.log('任务中断:' + message)
def make_handler(manager, token):
class Handler(BaseHTTPRequestHandler):
def log_message(self, *args):
pass
def send(self, status, payload, mime='application/json; charset=utf-8'):
data = json.dumps(payload, ensure_ascii=False).encode() if isinstance(payload, dict) else payload
self.send_response(status)
self.send_header('Content-Type', mime)
self.send_header('Content-Length', str(len(data)))
self.send_header('Cache-Control', 'no-store')
self.send_header('X-Content-Type-Options', 'nosniff')
self.end_headers()
self.wfile.write(data)
def local(self):
return self.headers.get('Host') in (f'127.0.0.1:{self.server.server_port}', f'localhost:{self.server.server_port}')
def do_GET(self):
if not self.local():
return self.send(403, {'error': '仅允许本机访问'})
path = urlsplit(self.path).path
if path == '/api/state':
return self.send(200, manager.snapshot())
if path == '/api/session':
return self.send(200, {'token': token})
files = {'/': ('index.html', 'text/html; charset=utf-8'), '/app.js': ('app.js', 'text/javascript; charset=utf-8'), '/style.css': ('style.css', 'text/css; charset=utf-8')}
if path not in files:
return self.send(404, {'error': '未找到页面'})
name, mime = files[path]
self.send(200, (ROOT / 'static' / name).read_bytes(), mime)
def do_POST(self):
if not self.local() or self.headers.get('X-Session-Token') != token:
return self.send(403, {'error': '页面会话失效,请刷新页面'})
key = ''
try:
length = int(self.headers.get('Content-Length', 0))
if not 0 < length <= 16384:
raise ValueError('请求大小无效')
body = json.loads(self.rfile.read(length))
if not isinstance(body, dict):
raise ValueError('请求必须是 JSON 对象')
if self.path == '/api/stop':
manager.stop.set()
return self.send(200, {'ok': True})
key = str(body.get('key') or '').strip()
if not key or len(key) > 512:
raise ValueError('请填写有效 API Key')
if self.path == '/api/channels':
channels = result_list(manager.requester(key, 'channel'))
if not channels or not all(isinstance(c, str) and c for c in channels):
raise ValueError('频道列表为空或格式错误')
return self.send(200, {'channels': channels})
if self.path == '/api/start':
target = body.get('target')
channel = body.get('channel')
if type(target) is not int or not 1 <= target <= 100000:
raise ValueError('下载数量须为 1 至 100000 的整数')
if not isinstance(channel, str) or not channel.strip() or len(channel) > 100:
raise ValueError('请选择新闻频道')
manager.start(key, channel, target)
return self.send(200, {'ok': True})
if self.path == '/api/debug':
endpoint = body.get('endpoint')
params = {}
if endpoint == 'index':
for field, low, high in [('num', 1, 40), ('start', 0, 400)]:
value = body.get(field)
if type(value) is not int or not low <= value <= high:
raise ValueError(f'{field} 必须在 {low}–{high} 之间')
params[field] = value
params['type'] = str(body.get('channel') or '')
before = time.monotonic()
payload = manager.requester(key, endpoint, params)
# 服务端响应同样隐藏 key,且只作为文本展示,不渲染上游 HTML。
clean = json.loads(json.dumps(payload, ensure_ascii=False).replace(key, '[已隐藏]'))
return self.send(200, {'elapsed': round(time.monotonic() - before, 2), 'request': API + str(endpoint) + '?' + urlencode({'key': '[已隐藏]', **params}), 'response': clean})
self.send(404, {'error': '未知操作'})
except Exception as exc:
message = str(exc)
if key:
message = message.replace(key, '[已隐藏]')
self.send(400, {'error': message[:500]})
return Handler
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--port', type=int, default=8765)
parser.add_argument('--no-browser', action='store_true')
args = parser.parse_args()
manager = Downloader()
server = ThreadingHTTPServer(('127.0.0.1', args.port), make_handler(manager, secrets.token_urlsafe(32)))
url = f'http://127.0.0.1:{server.server_port}'
print(f'新闻下载器:{url}\n保存目录:{manager.root}\n按 Ctrl+C 退出', flush=True)
if not args.no_browser:
webbrowser.open(url)
try:
server.serve_forever()
except KeyboardInterrupt:
manager.stop.set()
if manager.thread:
manager.thread.join(timeout=45)
finally:
server.server_close()
if __name__ == '__main__':
main()
+212
View File
@@ -0,0 +1,212 @@
#!/usr/bin/env python3
"""将探数新闻正文保存为与微信下载器相同的 Markdown 目录格式。"""
from __future__ import annotations
import base64
from dataclasses import asdict, dataclass
from pathlib import Path
import re
import shutil
import tempfile
from urllib.parse import unquote, urljoin, urlsplit
import requests
from bs4 import BeautifulSoup
from markdownify import markdownify
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
class ArticleDownloadError(RuntimeError):
"""页面不是文章、图片下载失败等可向调用方报告的错误。"""
@dataclass
class ArticleResult:
title: str
source_url: str
directory: str
markdown: str
images: list[str]
warnings: list[str]
def to_dict(self) -> dict:
return asdict(self)
def safe_name(title: str) -> str:
"""跨平台目录/文件名;目录名与其中 Markdown 的文件名保持一致。"""
name = re.sub(r'[<>:"/\\|?*\x00-\x1f\x7f]', '_', title)
name = re.sub(r'\s+', ' ', name).strip(' .')
# 同时限制 UTF-8 字节数,给 .md 和重名后缀保留空间。
name = name.encode('utf-8')[:180].decode('utf-8', errors='ignore').rstrip(' .')
if not name:
name = '未命名文章'
if re.fullmatch(r'(?i:CON|PRN|AUX|NUL|COM[1-9]|LPT[1-9])(?:\..*)?', name):
name = '_' + name
return name
def _session() -> requests.Session:
session = requests.Session()
session.headers.update({
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) '
'AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36',
'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8',
})
retry = Retry(total=3, backoff_factor=0.6,
status_forcelist=[429, 500, 502, 503, 504], allowed_methods=['GET'])
session.mount('https://', HTTPAdapter(max_retries=retry))
session.mount('http://', HTTPAdapter(max_retries=retry))
return session
def _get(session: requests.Session, url: str, timeout: float, limit: int,
referer: str | None = None) -> tuple[bytes, str, str]:
if urlsplit(url).scheme not in ('http', 'https'):
raise ArticleDownloadError(f'不支持的下载地址:{url[:200]}')
headers = {'Referer': referer} if referer else {}
with session.get(url, timeout=(min(timeout, 15), timeout),
headers=headers, stream=True) as response:
response.raise_for_status()
chunks, size = [], 0
for chunk in response.iter_content(65536):
size += len(chunk)
if size > limit:
raise ArticleDownloadError(f'文件超过大小限制({limit // 1024 // 1024} MB):{url[:200]}')
chunks.append(chunk)
return b''.join(chunks), response.headers.get('Content-Type', ''), response.url
def _image_ext(data: bytes, mime: str) -> str:
if data.startswith(b'\xff\xd8\xff'):
return '.jpg'
if data.startswith(b'\x89PNG\r\n\x1a\n'):
return '.png'
if data.startswith((b'GIF87a', b'GIF89a')):
return '.gif'
if data[:4] == b'RIFF' and data[8:12] == b'WEBP':
return '.webp'
if len(data) >= 12 and data[4:8] == b'ftyp' and data[8:12] in (b'avif', b'avis'):
return '.avif'
if b'<svg' in data[:1024].lower() and mime.split(';')[0] == 'image/svg+xml':
return '.svg'
raise ArticleDownloadError(f'返回内容不是支持的图片(Content-Type: {mime or "未知"})')
def _image_source(img) -> str:
for attr in ('data-src', 'data-original', 'data-actualsrc', 'data-lazy-src', 'src'):
value = img.get(attr)
if value and str(value).strip():
return str(value).strip()
srcset = img.get('data-srcset') or img.get('srcset')
if srcset:
return str(srcset).split(',')[-1].strip().split()[0]
return ''
def save_article(article: dict, output_root: str | Path = '.', *, timeout: float = 30,
allow_missing_images: bool = False) -> ArticleResult:
"""从 API 正文保存文章;图片失败时清理半成品,不覆盖已有同名目录。"""
url = str(article.get('weburl') or article.get('url') or '')
title = str(article.get('title') or '').strip()
raw_content = str(article.get('content') or '').strip()
if not title or not raw_content:
raise ArticleDownloadError('新闻标题或正文为空')
if timeout <= 0:
raise ValueError('timeout 必须大于 0')
with _session() as session:
final_url = url
soup = BeautifulSoup('<article>' + raw_content + '</article>', 'html.parser')
content = soup.article
for tag in content.select('script, style, noscript, iframe, form, button'):
tag.decompose()
# 部分微信排版把插图放在行内 CSS 背景中,也转为普通图片参与下载。
for tag in list(content.find_all(style=True)):
for match in re.finditer(r'url\(\s*[\'\"]?(.*?)[\'\"]?\s*\)', str(tag['style'])):
source = match.group(1).strip()
if source and not source.startswith('#'):
background = soup.new_tag('img', src=source, alt='')
tag.insert(0, background)
if not content.get_text(strip=True) and not content.find('img'):
raise ArticleDownloadError('文章正文为空,可能需要浏览器执行 JavaScript')
for link in content.find_all('a', href=True):
href = str(link['href']).strip()
if href and not href.startswith('#'):
absolute = urljoin(final_url, href)
if urlsplit(absolute).scheme in ('http', 'https', 'mailto', 'tel'):
link['href'] = absolute
else:
del link['href']
root = Path(output_root).expanduser().resolve()
root.mkdir(parents=True, exist_ok=True)
stage = Path(tempfile.mkdtemp(prefix='.article-', dir=root))
images, warnings, downloaded = [], [], {}
try:
for img in list(content.find_all('img')):
source = _image_source(img)
if not source:
img.replace_with(soup.new_string('[图片缺少下载地址]'))
if not allow_missing_images:
raise ArticleDownloadError('正文中存在没有下载地址的图片')
warnings.append('正文中存在没有下载地址的图片')
continue
image_url = urljoin(final_url, source)
try:
if image_url not in downloaded:
if source.startswith('data:image/'):
header, payload = source.split(',', 1)
data = base64.b64decode(payload, validate=True) if ';base64' in header else unquote(payload).encode()
mime = header[5:].split(';')[0]
if len(data) > 50 * 1024 * 1024:
raise ArticleDownloadError('内嵌图片超过 50 MB')
else:
data, mime, _ = _get(session, image_url, timeout, 50 * 1024 * 1024, final_url)
ext = _image_ext(data, mime)
filename = f'image{len(images) + 1:03d}{ext}'
(stage / filename).write_bytes(data)
images.append(filename)
downloaded[image_url] = filename
# 清除懒加载/响应式属性,使转换器只引用本地文件。
alt = str(img.get('alt') or '')
img.attrs = {'src': downloaded[image_url], 'alt': alt}
except (requests.RequestException, ArticleDownloadError, ValueError) as exc:
if not allow_missing_images:
raise ArticleDownloadError(f'图片下载失败:{image_url[:200]};{exc}') from exc
warnings.append(f'图片下载失败:{image_url[:200]};{exc}')
img.replace_with(soup.new_string(f'[图片下载失败:{img.get("alt") or "未命名图片"}]'))
# 新闻图片后常带空的图注标题,避免生成单独的 ## 和多余空行。
for heading in content.select('h1, h2, h3, h4, h5, h6'):
if not heading.get_text(strip=True) and not heading.find('img'):
heading.decompose()
body = markdownify(str(content), heading_style='ATX', bullets='-',
strip=['picture', 'source']).strip()
# 微信富文本中各段常没有标准 p 标签;markdownify 仍会保留文字和图片顺序。
author = str(article.get('src') or '未知作者').strip() or '未知作者'
text = f'# {title}\n\n'
text += body + f'\n\n来源:{author},如有侵权,请联系删除。\n'
basename = safe_name(title)
# 原子认领新目录,支持多个进程并发保存同名文章。
index = 1
while True:
name = basename if index == 1 else f'{basename} ({index})'
destination = root / name
try:
destination.mkdir()
break
except FileExistsError:
index += 1
try:
(stage / f'{name}.md').write_text(text, encoding='utf-8')
for item in stage.iterdir():
shutil.move(str(item), destination / item.name)
except BaseException:
shutil.rmtree(destination, ignore_errors=True)
raise
return ArticleResult(title, url, str(destination), str(destination / f'{name}.md'),
[str(destination / x) for x in images], warnings)
finally:
shutil.rmtree(stage, ignore_errors=True)
+3
View File
@@ -0,0 +1,3 @@
requests>=2.32,<3
beautifulsoup4>=4.12,<5
markdownify>=1.1,<2
+79
View File
@@ -0,0 +1,79 @@
'use strict';
const $ = id => document.getElementById(id);
let token = '', running = false, starting = false, loadingChannels = false, loadedKey = '', lastLog = -1;
const labels = {idle:'等待开始',running:'下载中',completed:'已完成',partial:'未达到目标',cancelled:'已停止',error:'任务中断'};
const key = () => $('key').value.trim();
function controls() {
const busy = running || starting;
$('start').disabled = busy || !loadedKey || loadedKey !== key() || !$('channel').value;
$('stop').disabled = !running;
$('channels').disabled = busy || loadingChannels;
for (const id of ['key','channel','target']) $(id).disabled = busy || (id === 'channel' && !loadedKey);
}
async function post(path, body) {
const response = await fetch('/api/' + path, {method:'POST',headers:{'Content-Type':'application/json','X-Session-Token':token},body:JSON.stringify(body)});
const data = await response.json();
if (!response.ok) throw new Error(data.error || '请求失败');
return data;
}
$('key').addEventListener('input', () => {loadedKey=''; $('channel').replaceChildren(new Option('请重新获取频道','')); controls();});
$('channels').addEventListener('click', async () => {
loadingChannels=true; controls(); $('notice').textContent='正在获取频道…';
const submittedKey = key();
try {
const data = await post('channels',{key:submittedKey});
if (key() !== submittedKey) return;
$('channel').replaceChildren(...data.channels.map(c => new Option(c,c)));
loadedKey=submittedKey; $('notice').textContent=`已获取 ${data.channels.length} 个频道。`;
$('debug-channel').value=$('channel').value;
} catch(error) {$('notice').textContent=error.message;}
finally {loadingChannels=false; controls();}
});
$('channel').addEventListener('change', () => {$('debug-channel').value=$('channel').value; controls();});
$('download-form').addEventListener('submit', async event => {
event.preventDefault(); starting=true; controls(); $('notice').textContent='';
try {await post('start',{key:key(),channel:$('channel').value,target:Number($('target').value)}); await refresh();}
catch(error) {$('notice').textContent=error.message;}
finally {starting=false; controls();}
});
$('stop').addEventListener('click', async () => {
$('stop').disabled=true;
try {await post('stop',{}); $('notice').textContent='正在停止,等待当前请求或文章保存结束…';}
catch(error) {$('notice').textContent=error.message;}
});
$('debug-form').addEventListener('submit', async event => {
event.preventDefault(); $('debug-submit').disabled=true; $('debug-output').textContent='正在请求…';
try {
const data=await post('debug',{key:key(),endpoint:$('endpoint').value,channel:$('debug-channel').value,num:Number($('debug-num').value),start:Number($('debug-start').value)});
const output=JSON.stringify(data,null,2);
$('debug-output').textContent=output.slice(0,100000)+(output.length>100000?'\n…输出超过 100000 字符,已截断。':'');
} catch(error) {$('debug-output').textContent=error.message;}
finally {$('debug-submit').disabled=false;}
});
async function refresh() {
const response=await fetch('/api/state');
if (!response.ok) throw new Error('无法读取下载状态');
const data=await response.json();
running=data.status==='running'; controls();
const percent=data.target ? Math.floor(data.saved/data.target*100) : 0;
$('status').textContent=labels[data.status] || data.status;
$('progress-text').textContent=`下载新闻 ${data.saved}/${data.target} 条`;
$('percent').textContent=`已完成 ${percent}%`; $('progress').value=percent;
for (const id of ['saved','skipped','failed','requests','message','output']) $(id).textContent=data[id];
const latest=data.logs.length ? data.logs[data.logs.length-1].id : 0;
if (latest!==lastLog) {
const logs=$('logs'); const bottom=logs.scrollHeight-logs.scrollTop-logs.clientHeight<40;
logs.replaceChildren(...data.logs.map(log => {const row=document.createElement('div'); const stamp=document.createElement('time'); stamp.textContent=log.time; row.append(stamp,document.createTextNode(log.message)); return row;}));
if (bottom) logs.scrollTop=logs.scrollHeight;
lastLog=latest;
}
}
async function poll() {
try {await refresh();}
catch(error) {$('notice').textContent='与本地服务连接中断,请检查启动窗口。';}
setTimeout(poll,700);
}
(async () => {
try {const response=await fetch('/api/session'); token=(await response.json()).token; await poll();}
catch(error) {$('notice').textContent='初始化失败,请刷新页面。';}
})();
+14
View File
@@ -0,0 +1,14 @@
<!doctype html>
<html lang="zh-CN">
<head><meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1"><title>探数 · 新闻下载器</title><link rel="stylesheet" href="/style.css"><script src="/app.js" defer></script></head>
<body><main>
<header><div class="brand">TANSHU / NEWS ARCHIVE</div><h1>把新闻,保存到本地。</h1><p>选择频道,收集新闻。正文与图片一起保存,自动跳过已下载的内容。</p></header>
<section class="card"><div class="section-heading"><h2>下载设置</h2><span class="badge">本地运行</span></div>
<form id="download-form"><label for="key">API Key</label><div class="key-row"><input id="key" type="password" placeholder="填写探数 API Key" autocomplete="off" required maxlength="512"><button id="channels" type="button" class="secondary">获取新闻频道</button></div><p class="hint">密钥仅用于本次页面会话,不写入文件或浏览器存储。</p>
<div class="fields"><div><label for="channel">新闻频道</label><select id="channel" required disabled><option value="">先填写密钥并获取频道</option></select></div><div><label for="target">下载数量</label><input id="target" type="number" min="1" max="100000" step="1" value="10" required></div></div>
<p class="hint">接口每次最多 40 条,起始位置最多 400。遇到重复会继续向后查找;可用新闻不足时会说明原因。</p><div class="actions"><button id="start" disabled>开始下载</button><button id="stop" type="button" class="secondary" disabled>停止下载</button></div><p id="notice" role="alert"></p></form></section>
<section class="card"><div class="section-heading"><h2>下载进度</h2><span id="status" class="badge">等待开始</span></div><div class="progress-heading"><strong id="progress-text">下载新闻 0/0 条</strong><span id="percent">已完成 0%</span></div><progress id="progress" max="100" value="0" aria-label="下载进度"></progress><div class="stats"><span>已保存 <b id="saved">0</b></span><span>重复跳过 <b id="skipped">0</b></span><span>保存失败 <b id="failed">0</b></span><span>接口请求 <b id="requests">0</b></span></div><p id="message">等待开始</p><p class="hint path">保存位置:<span id="output">加载中…</span></p></section>
<section class="card"><div class="section-heading"><h2>运行输出</h2><span class="hint">保留最近 300 条</span></div><div id="logs" class="console" role="log" aria-label="运行输出"><div class="empty">运行情况将在这里显示。</div></div></section>
<details class="card"><summary>调试终端 <span>手动请求接口</span></summary><p class="hint">使用上方 API Key。请求会消耗接口调用额度,但不会保存新闻。</p><form id="debug-form"><div class="fields"><div><label for="endpoint">请求接口</label><select id="endpoint"><option value="channel">新闻频道查询 /channel</option><option value="index">新闻列表查询 /index</option></select></div><div><label for="debug-channel">type · 频道</label><input id="debug-channel" value="头条"></div><div><label for="debug-num">num · 数量(1–40)</label><input id="debug-num" type="number" min="1" max="40" value="1" required></div><div><label for="debug-start">start · 起始位置(0–400)</label><input id="debug-start" type="number" min="0" max="400" value="0" required></div></div><button id="debug-submit" class="secondary">发送测试请求</button></form><pre id="debug-output" class="console">尚未发送请求。</pre></details>
<footer>Markdown 正文 · 本地图片 · 历史去重</footer>
</main></body></html>
+1
View File
@@ -0,0 +1 @@
:root{font-family:Inter,-apple-system,BlinkMacSystemFont,"Segoe UI","PingFang SC",sans-serif;color:#20312e;background:#f3f6f3;font-size:15px}*{box-sizing:border-box}body{margin:0}main{max-width:960px;margin:0 auto;padding:48px 24px 24px}header{margin-bottom:32px}.brand{font-size:12px;letter-spacing:2px;font-weight:700;color:#437b69}h1{font-size:36px;letter-spacing:-1px;margin:14px 0}header p{color:#687a74;line-height:1.8}h2{font-size:17px;margin:0}.card{padding:26px 28px;background:#fff;border:1px solid #dfe7e1;border-radius:16px;margin-bottom:20px;box-shadow:0 4px 16px #243c2904}.section-heading{display:flex;align-items:center;justify-content:space-between;gap:10px;margin-bottom:24px}.badge{background:#edf4ef;color:#4c7564;padding:6px 10px;border-radius:20px;font-size:12px}.key-row{display:flex;gap:12px}.key-row input{flex:1;min-width:0}label{display:block;font-weight:600;margin-bottom:9px;font-size:13px}input,select,button{font:inherit;border-radius:8px}input,select{background:#fbfcfb;border:1px solid #d7e0d9;width:100%;padding:12px;color:inherit}input:focus,select:focus,button:focus-visible,summary:focus-visible{outline:3px solid #8dcab5;outline-offset:2px}button{cursor:pointer;border:1px solid #286850;background:#286850;color:#fff;padding:12px 22px;font-weight:600;white-space:nowrap}.secondary{background:#fff;color:#345e4f;border-color:#cedbd3}button:disabled{opacity:.45;cursor:not-allowed}.fields{display:grid;grid-template-columns:1fr 1fr;gap:20px;margin:24px 0 16px}.hint{font-size:12px;color:#79867f;line-height:1.8;margin:10px 0}.actions{display:flex;gap:12px;margin-top:22px}#notice{color:#ac4c32;font-size:13px;white-space:pre-wrap}#notice:empty{display:none}.progress-heading{display:flex;justify-content:space-between;gap:16px;font-size:14px}#percent{color:#3f7b60}progress{display:block;width:100%;height:12px;border:0;border-radius:10px;overflow:hidden;margin:18px 0 20px;background:#edf2ef}progress::-webkit-progress-bar{background:#edf2ef}progress::-webkit-progress-value{background:#3c8b68;transition:width .25s}progress::-moz-progress-bar{background:#3c8b68}.stats{display:flex;gap:28px;flex-wrap:wrap;color:#7a8880;font-size:13px}.stats b{color:#304e40;margin-left:8px;font-size:17px}#message{font-size:13px;color:#52685b;line-height:1.8}.path{overflow-wrap:anywhere}.console{height:260px;overflow:auto;background:#15251f;color:#c5dacf;border-radius:10px;padding:18px;font:12px/1.9 ui-monospace,SFMono-Regular,Consolas,monospace;white-space:pre-wrap;overflow-wrap:anywhere}.console time{color:#7eac94;margin-right:14px}.empty{color:#7e9889}summary{cursor:pointer;font-weight:600}summary span{font-size:12px;font-weight:400;color:#79867f;margin-left:12px}#debug-output{margin-bottom:0;height:300px}footer{text-align:center;color:#89958e;font-size:12px;margin:32px 0}@media(max-width:600px){main{padding:26px 14px}h1{font-size:28px}.card{padding:22px 18px}.key-row{flex-direction:column}.fields{grid-template-columns:1fr}.stats{gap:14px}.progress-heading{font-size:12px}}
+63
View File
@@ -0,0 +1,63 @@
#!/usr/bin/env python3
"""Build a portable v1 article package for the DWW11 userscript (standard library only)."""
from __future__ import annotations
import argparse
import base64
import json
import mimetypes
from pathlib import Path
def make_package(directory: str | Path, markdown: str | None = None) -> dict:
"""Return JSON-serializable data; no network access or publishing side effects."""
root = Path(directory).expanduser().resolve(strict=True)
if not root.is_dir():
raise ValueError("文章目录不存在")
choices = sorted(path for path in root.rglob("*") if path.is_file() and path.suffix.lower() == ".md")
if markdown:
selected = (root / markdown).resolve(strict=True)
elif len(choices) == 1:
selected = choices[0].resolve()
else:
raise ValueError("目录必须包含一个 Markdown 文件,或通过 --markdown 指定目录内相对路径")
if not selected.is_relative_to(root) or selected.suffix.lower() != ".md" or not selected.is_file():
raise ValueError("Markdown 必须位于所选文章目录内")
images = []
# Package local supported images. The importer uploads only those referenced by Markdown.
for path in sorted(root.rglob("*")):
if not path.is_file() or path.suffix.lower() not in {".jpg", ".jpeg", ".png", ".gif"}:
continue
if not path.resolve().is_relative_to(root):
raise ValueError(f"图片符号链接指向目录外:{path.name}")
if not 0 < path.stat().st_size < 20 * 1024 * 1024:
raise ValueError(f"图片为空或达到 20 MB:{path.name}")
images.append({"path": path.relative_to(root).as_posix(),
"mime": mimetypes.guess_type(path.name)[0] or "application/octet-stream",
"base64": base64.b64encode(path.read_bytes()).decode("ascii")})
return {"schemaVersion": 1, "markdownPath": selected.relative_to(root).as_posix(),
"markdown": selected.read_text(encoding="utf-8-sig"), "images": images}
def main() -> None:
parser = argparse.ArgumentParser(description="把文章目录打包为油猴脚本可导入的 JSON;不上传或发布")
parser.add_argument("directory", help="包含 Markdown 和图片的目录")
parser.add_argument("--markdown", help="多个 Markdown 时指定目录内的相对路径")
parser.add_argument("-o", "--output", help="输出文件路径;未指定时输出到标准输出")
args = parser.parse_args()
try:
content = json.dumps(make_package(args.directory, args.markdown), ensure_ascii=False, indent=2)
if args.output:
output = Path(args.output).expanduser()
# Never silently overwrite an article or an existing package.
with output.open("x", encoding="utf-8") as handle:
handle.write(content + "\n")
print(json.dumps({"ok": True, "package": str(output.resolve())}, ensure_ascii=False))
else:
print(content)
except (OSError, ValueError) as error:
parser.exit(1, f"打包失败:{error}\n")
if __name__ == "__main__":
main()
+3
View File
@@ -0,0 +1,3 @@
requests>=2.32,<3
beautifulsoup4>=4.12,<5
markdownify>=1.1,<2
Executable
+8
View File
@@ -0,0 +1,8 @@
#!/bin/zsh
cd "${0:A:h}"
if ! command -v uv >/dev/null 2>&1; then
print '未找到 uv,请安装 uv 后重试。'
read '?按回车关闭…'
exit 1
fi
uv run --no-project --python 3.12 --with-requirements requirements.txt python app.py
+51
View File
@@ -0,0 +1,51 @@
'use strict';
const $=id=>document.getElementById(id);
let token='',state=null,loadedKey='',refreshing=false;
const labels={queued:'待处理',preparing:'准备中',importing:'上传与导入',formatting:'排版中',awaiting_user_review:'等待人工审核',user_submitting:'已观察到人工提交',submit_unknown:'发布结果待核对',accepted_pending_moderation:'已受理 · 审核中',published:'已确认公开',needs_attention:'需要处理',cancelled:'已移出'};
function notice(message){$('notice').hidden=!message;$('notice').textContent=message;}
async function api(route,body={}) {
const response=await fetch('/api/'+route,{method:'POST',headers:{'Content-Type':'application/json','X-Session-Token':token},body:JSON.stringify(body)});
const data=await response.json();if(!response.ok)throw new Error(data.error||'请求失败');return data;
}
function action(id,fn){$(id).addEventListener('click',async()=>{const button=$(id);button.disabled=true;try{await fn();await refresh();}catch(error){notice(error.message);}finally{button.disabled=false;}});}
function text(tag,value,className){const e=document.createElement(tag);e.textContent=value;if(className)e.className=className;return e;}
function render(data){
state=data;const {jobs,run,download}=data;
$('total').textContent=jobs.length;$('review').textContent=jobs.filter(j=>j.state==='awaiting_user_review').length;
$('accepted').textContent=jobs.filter(j=>['published','accepted_pending_moderation'].includes(j.state)).length;
$('batch').textContent=run.batch?`${run.batch.count} / ${run.batch.target}`:'—';$('account').textContent=run.batch?run.batch.account.name:'尚未绑定账号';
$('run-state').textContent=run.running?'流程运行中':'流程已暂停';$('run-reason').textContent=run.reason;
$('run-light').classList.toggle('gray',!run.running);$('empty').hidden=jobs.length>0;
$('jobs').replaceChildren(...jobs.map(job=>{
const tr=document.createElement('tr'),title=document.createElement('td');title.append(text('div',job.title),text('small',`${job.image_count} 张配图 · ${job.source}`));if(job.original_title!==job.title)title.append(text('small','已缩短标题,原题:'+job.original_title));
const status=document.createElement('td');status.append(text('span',labels[job.state]||job.state,'badge'));if(job.error)status.append(text('small',job.error));
const account=text('td',job.account?`${job.account.name} / ${job.fid}`:'待领取');const result=document.createElement('td');
if(job.post_url){const link=text('a','查看帖子');link.href=job.post_url;link.target='_blank';link.rel='noopener';result.append(link);}
if(!['published','accepted_pending_moderation'].includes(job.state)){
const retry=text('button','人工核对 / 处理');retry.className='secondary';retry.onclick=async()=>{
if(state.run.running){notice('请先暂停流程,再处理任务');return;}
if(!confirm('只有确认网站未受理这篇文章,且已清理编辑器/草稿后才能继续。\n已受理、审核中或结果不明的文章请取消。\n你是否已经确认未受理?'))return;
const again=confirm('重新加入队列?\n确定:重新排队。取消:移出队列。');
try{await api('resolve',{id:job.id,confirmed_not_accepted:true,retry:again});await refresh();}catch(error){notice(error.message);}
};result.append(retry);
}
tr.append(title,status,account,result);return tr;
}));
const dlLabels={idle:'等待开始',running:'下载中',completed:'下载完成',partial:'未达到目标',cancelled:'已停止',error:'下载出错'};
$('download-state').textContent=dlLabels[download.status]||download.status;$('download-progress').textContent=`已保存 ${download.saved}/${download.target} 篇 · 跳过 ${download.skipped} · 失败 ${download.failed}`;
$('progress').value=download.target?Math.floor(download.saved/download.target*100):0;
$('download').disabled=download.status==='running';$('logs').textContent=download.logs.map(x=>`${x.time} ${x.message}`).join('\n')||'暂无日志';
$('browser-state').textContent=data.clients.length?`${data.clients.length} 个标签在线`:'尚未连接';
$('clients').textContent=data.clients.map(c=>`${c.account?.name||'未识别账号'} · 换号插件${c.extension?.ready?'已就绪':'未连接或需更新'}`).join('\n')||'没有在线执行标签';
}
async function refresh(){if(refreshing||!token)return;refreshing=true;try{render(await api('state'));}catch(error){notice(error.message);$('run-state').textContent='本地服务连接中断';}finally{refreshing=false;}}
action('pair',async()=>{const value=await api('pair');$('pair-code').hidden=false;$('pair-code').type='text';$('pair-code').value=value.token;$('pair-code').select();});
action('channels',async()=>{const value=await api('channels',{key:$('key').value.trim()});loadedKey=$('key').value.trim();$('channel').replaceChildren(...value.channels.map(c=>new Option(c,c)));});
$('key').oninput=()=>{loadedKey='';$('channel').replaceChildren(new Option('请重新获取频道',''));};
$('download-form').onsubmit=async event=>{event.preventDefault();try{if(!loadedKey||loadedKey!==$('key').value.trim())throw new Error('请先获取频道');await api('download',{key:loadedKey,channel:$('channel').value,target:Number($('target').value)});notice('下载已开始,完整保存的文章会自动入队');await refresh();}catch(error){notice(error.message);}};
action('download-stop',()=>api('download-stop'));
action('scan',async()=>{notice('正在检查文章目录…');const value=await api('scan',{root:$('root').value.trim()});$('scan-result').textContent=value.results.map(x=>`${x.directory}:${x.status}`).join('\n')||'目录中未找到文章';notice('目录检查完成');});
$('config-form').onsubmit=async event=>{event.preventDefault();try{await api('configure',{fid:Number($('fid').value),minimum:Number($('minimum').value),maximum:Number($('maximum').value),interval:Number($('interval').value),history_checked:$('history-checked').checked});notice('处理规则已保存');await refresh();}catch(error){notice(error.message);}};
for(const name of ['start','pause','stop'])action(name,async()=>{await api(name);notice(name==='stop'?'流程已终止;浏览器将在下一次连接检查时停止后续操作。已发出的请求可能仍已完成。':'');});
$('history-file').onchange=async()=>{try{const file=$('history-file').files[0];if(!file)return;if(file.size>2*1024*1024)throw new Error('记录文件过大');const value=JSON.parse(await file.text());await api('history',{records:value.records});notice('历史记录已导入;仅使用标记为 block_repost 的记录');await refresh();}catch(error){notice(error.message);}};
(async()=>{try{const response=await fetch('/api/session');const session=await response.json();if(!response.ok)throw new Error(session.error);token=session.token;$('port').value=location.port;await refresh();if(state?.run.config?.fid){for(const id of ['fid','minimum','maximum','interval'])$(id).value=state.run.config[id];$('history-checked').checked=state.run.config.history_checked;}}catch(error){notice(error.message);}setInterval(refresh,2500);})();
+26
View File
@@ -0,0 +1,26 @@
<!doctype html>
<html lang="zh-CN">
<head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"><title>新闻工作台</title><link rel="stylesheet" href="/style.css"></head>
<body>
<aside><div class="brand"><span class="mark">闻</span><div>新闻工作台<small>本地内容工作流</small></div></div><nav><a href="#overview" class="selected">工作概览</a><a href="#source">下载与文章</a><a href="#settings">处理规则</a><a href="#queue">文章队列</a><a href="#connection">浏览器连接</a></nav><div class="aside-note"><span class="dot"></span> 数据保存在本机<p>文章自动准备与排版<br>发表由你亲手完成</p></div></aside>
<main>
<header id="overview"><div><div class="eyebrow">CONTENT WORKSPACE</div><h1>把内容准备好,把审核留给你。</h1><p>下载、导入、排版与账号切换,在一个工作台管理。</p></div><button id="stop" class="danger">终止流程</button></header>
<div id="notice" role="status" aria-live="polite" class="notice" hidden></div>
<section class="flow"><div><span>01</span>下载新闻</div><i>→</i><div><span>02</span>导入与排版</div><i>→</i><div class="highlight"><span>03</span>人工审核与发表</div><i>→</i><div><span>04</span>结果核对与换号</div></section>
<div class="pending-note"><b>当前接入进度</b> 下载、导入、排版与换号接口已整合;结果页自动核对等待页面 HTML。人工发表后会暂停,不自动计数或继续下一篇。</div>
<section class="metrics"><article><small>队列文章</small><strong id="total">0</strong><span>完整文件检查后入队</span></article><article><small>等待你审核</small><strong id="review">0</strong><span>请在网站检查并亲手发表</span></article><article><small>明确受理</small><strong id="accepted">0</strong><span>审核中与已公开分别记录</span></article><article><small>当前账号批次</small><strong id="batch">—</strong><span id="account">尚未绑定账号</span></article></section>
<div class="workspace-grid">
<section id="source" class="card"><div class="section-head"><div><span class="eyebrow">SOURCE</span><h2>下载与文章</h2></div><span id="download-state" class="badge">等待开始</span></div>
<form id="download-form"><label>探数 API Key<input id="key" type="password" autocomplete="off" placeholder="仅在本次运行中使用"></label><div class="row"><label class="grow">新闻频道<select id="channel" required><option value="">先获取频道</option></select></label><button id="channels" type="button" class="secondary">获取频道</button></div><div class="row"><label class="grow">下载篇数<input id="target" type="number" min="1" max="100000" value="5" required></label><button id="download" type="submit">开始下载</button><button id="download-stop" type="button" class="secondary">停止下载</button></div></form>
<div class="progress"><progress id="progress" max="100" value="0"></progress><small id="download-progress">等待下载</small></div>
<details><summary>导入已有文章目录</summary><p class="muted">填写包含文章子文件夹的完整路径。留空时扫描本工具的 downloads 目录。请在文件完整保存后扫描。</p><label>文章根目录<input id="root" placeholder="/完整路径/新闻目录"></label><button id="scan" class="secondary">检查并加入队列</button><pre id="scan-result"></pre></details>
</section>
<section id="settings" class="card"><div class="section-head"><div><span class="eyebrow">WORKFLOW</span><h2>处理规则</h2></div><span class="badge">人工发表</span></div>
<form id="config-form"><label>目标版块 ID(fid)<input id="fid" type="number" min="1" placeholder="从新帖页地址中的 fid 获取" required></label><div class="row"><label class="grow">每账号最少篇数<input id="minimum" type="number" min="1" max="1000" placeholder="最小值" required></label><label class="grow">每账号最多篇数<input id="maximum" type="number" min="1" max="1000" placeholder="最大值" required></label></div><label>两篇之间间隔(秒)<input id="interval" type="number" min="0" max="86400" placeholder="请输入间隔" required></label><label class="check"><input id="history-checked" type="checkbox" required><span>我已核对待处理文章与历史发布记录,避免重复发表。</span></label><button type="submit" class="secondary">保存处理规则</button></form><p class="muted">每个账号只抽取一次目标篇数;只有明确受理的文章才计数。历史演示去重记录会自动读取。</p>
</section></div>
<section class="runbar"><div><span id="run-light" class="dot gray"></span><b id="run-state">正在连接本地服务…</b><p id="run-reason">等待状态</p></div><div class="actions"><button id="start">开始 / 继续</button><button id="pause" class="secondary">暂停</button></div></section>
<section id="queue" class="card"><div class="section-head"><div><span class="eyebrow">ARTICLE QUEUE</span><h2>文章队列</h2></div><label class="file-button">导入历史去重记录<input id="history-file" type="file" accept=".json,application/json" hidden></label></div><div class="table-wrap"><table><thead><tr><th>文章</th><th>状态</th><th>账号 / 版块</th><th>结果与操作</th></tr></thead><tbody id="jobs"></tbody></table><div id="empty" class="empty"><span>+</span><b>从第一篇文章开始</b><p>下载新闻,或检查已有文章目录。准备好的文章会出现在这里。</p></div></div></section>
<section id="connection" class="card"><div class="section-head"><div><span class="eyebrow">CONNECTION</span><h2>连接你的浏览器</h2></div><span id="browser-state" class="badge">尚未连接</span></div><div class="connection-grid"><div><ol><li>在油猴中安装以下两个脚本,并停用原导入、排版脚本。</li><li>重新加载原目录中的换号插件,然后刷新网站。</li><li>在网站左下角展开连接设置,填入本地端口和连接码。</li></ol><div class="actions"><a class="button secondary" href="/browser/local-bridge.user.js">安装本地连接脚本</a><a class="button secondary" href="/browser/auto-news.user.js">安装导入排版脚本</a></div></div><div><label>本地端口<input id="port" readonly></label><button id="pair" class="secondary">显示本次连接码</button><input id="pair-code" type="password" readonly hidden aria-label="本次连接码"><small class="muted">服务重启后需要更新连接码。仅复制到你安装的本地连接脚本。</small><p id="clients" class="muted">没有在线执行标签</p></div></div></section>
<details class="card"><summary>下载日志</summary><pre id="logs">暂无日志</pre></details><footer>新闻工作台 · 本机运行 · v0.1 &nbsp; / &nbsp; 终止后不会自动恢复,已发出的请求可能仍已完成。</footer>
</main><script src="/app.js"></script>
</body></html>
+1
View File
@@ -0,0 +1 @@
:root{font-family:Inter,-apple-system,BlinkMacSystemFont,"PingFang SC","Microsoft YaHei",sans-serif;color:#203b38;background:#f3f6f4;font-synthesis:none;font-size:14px}*{box-sizing:border-box}body{margin:0}aside{position:fixed;inset:0 auto 0 0;width:220px;background:#113e37;color:#f4fbf7;padding:32px 22px;display:flex;flex-direction:column}.brand{display:flex;align-items:center;gap:12px;font-size:20px;font-weight:650;letter-spacing:.5px}.mark{display:grid;place-items:center;background:#d8efb9;color:#17483e;width:42px;height:42px;border-radius:13px}.brand small{display:block;font-size:11px;font-weight:400;color:#b3cdc5;margin-top:5px}nav{display:grid;gap:9px;margin-top:54px}nav a{color:#bad1c9;text-decoration:none;padding:12px 16px;border-radius:8px}nav a:hover,nav .selected{background:#28534a;color:#fff}.aside-note{margin-top:auto;color:#bad1c9;font-size:12px;line-height:1.9}.dot{display:inline-block;width:8px;height:8px;border-radius:50%;background:#a3db80;margin-right:7px}.dot.gray{background:#a1ada7}main{max-width:1660px;margin-left:220px;padding:42px 40px}header{display:flex;align-items:center;justify-content:space-between;gap:20px}.eyebrow{font-size:10px;letter-spacing:2px;color:#6b8980;font-weight:700}h1{font-size:28px;letter-spacing:-.6px;margin:12px 0}header p{color:#73857e;margin:0;font-size:13px}button,.button,.file-button{font:inherit;font-weight:550;border:1px solid #1c6554;border-radius:7px;background:#1c6554;color:white;padding:10px 15px;cursor:pointer;white-space:nowrap;text-decoration:none;display:inline-block}button:hover,.button:hover{filter:brightness(.94)}button:disabled{opacity:.45;cursor:wait}.secondary,.file-button{background:white;color:#36574d;border-color:#d4dfd7}.danger{background:#fff8f6;border-color:#e7c5bb;color:#a34d37}.flow{display:flex;align-items:center;justify-content:space-between;gap:12px;margin-top:34px;background:white;border:1px solid #e2e9e2;padding:22px 25px;border-radius:12px;font-size:12px}.flow span{background:#edf3ef;color:#5f7d70;padding:5px 7px;border-radius:5px;font-size:10px;margin-right:8px}.flow i{color:#b1c0b6;font-style:normal}.flow .highlight{font-weight:650;color:#2e6947}.flow .highlight span{background:#e3efcc;color:#476635}.pending-note{background:#f8f2df;border:1px solid #eae0bd;color:#7b6938;padding:12px 16px;border-radius:8px;margin:16px 0;font-size:12px;line-height:1.8}.pending-note b{margin-right:10px}.metrics{display:grid;grid-template-columns:repeat(4,1fr);gap:16px;margin:22px 0}.metrics article{padding:20px 23px;background:white;border:1px solid #e0e8e0;border-radius:10px}.metrics small{font-size:12px;color:#6b7f73}.metrics strong{display:block;font-size:30px;font-weight:550;margin:12px 0 8px}.metrics span{font-size:11px;color:#8a988f}.workspace-grid{display:grid;grid-template-columns:1.1fr 1fr;gap:20px}.card{background:white;border:1px solid #e0e8e0;border-radius:12px;padding:24px;margin-bottom:20px;min-width:0}.section-head{display:flex;align-items:center;justify-content:space-between;gap:12px;margin-bottom:20px}h2{font-size:17px;margin:6px 0 0;font-weight:600}.badge{font-size:11px;border-radius:20px;background:#f0f5ef;color:#6c826d;padding:5px 10px}.row{display:flex;gap:12px;align-items:flex-end}.grow{flex:1;min-width:0}label{display:block;font-size:12px;color:#61756b;margin-bottom:14px}input,select{display:block;width:100%;font:inherit;color:#304a3f;padding:10px 12px;margin-top:7px;border:1px solid #dbe4dd;border-radius:6px;background:#fcfdfb;outline:none}input:focus,select:focus{border-color:#659981;box-shadow:0 0 0 3px #6599811a}input::placeholder{color:#a4afa8}.row button{margin-bottom:14px}.check{display:flex;align-items:flex-start;gap:8px;line-height:1.7}.check input{width:15px;height:15px;margin:2px 0 0;accent-color:#276b54}.muted{font-size:12px;line-height:1.8;color:#8b988e}.progress{margin:12px 0 20px}progress{appearance:none;width:100%;height:5px;border:0;border-radius:4px;display:block;background:#edf2ec}progress::-webkit-progress-bar{background:#edf2ec}progress::-webkit-progress-value{background:#76a77f;border-radius:4px}.progress small{display:block;color:#82917f;margin-top:8px;font-size:11px}summary{cursor:pointer;color:#617d6d;font-size:12px}details label{margin-top:12px}.runbar{display:flex;align-items:center;justify-content:space-between;border-radius:10px;background:#e5eee5;padding:19px 23px;margin-bottom:22px;gap:16px}.runbar b{font-weight:550}.runbar p{margin:7px 0 0 15px;font-size:12px;color:#758b76}.actions{display:flex;gap:10px;flex-wrap:wrap}table{border-collapse:collapse;width:100%;text-align:left;font-size:12px}th{color:#8a998f;font-weight:500;background:#f7f9f5;padding:12px}td{padding:16px 12px;border-bottom:1px solid #eef1eb;vertical-align:top;line-height:1.8}td:first-child{max-width:350px}td small{display:block;color:#91a093;font-size:11px}td button{font-size:11px;padding:4px 8px}td a{color:#1c6554}.table-wrap{overflow-x:auto}.empty{padding:44px 16px;text-align:center;color:#81927f}.empty span{display:block;foLine truncated
+294
View File
@@ -0,0 +1,294 @@
"""Persistent serial queue. Browser submissions are always initiated by the user."""
import json
from contextlib import contextmanager
from pathlib import Path
import secrets
import sqlite3
import threading
import time
from urllib.parse import urlsplit
from articles import inspect_article
ACCEPTED = {'published', 'accepted_pending_moderation'}
SUBMITTED = {'user_submitting', 'submit_unknown'}
class Store:
def __init__(self, path, ledger=None):
self.path = str(path)
Path(path).parent.mkdir(parents=True, exist_ok=True)
self.lock = threading.RLock()
with self.db() as db:
db.executescript('''
CREATE TABLE IF NOT EXISTS jobs(id TEXT PRIMARY KEY, article_key TEXT UNIQUE, value TEXT NOT NULL);
CREATE TABLE IF NOT EXISTS ledger(key TEXT PRIMARY KEY, value TEXT NOT NULL);
CREATE TABLE IF NOT EXISTS settings(id INTEGER PRIMARY KEY CHECK(id=1), value TEXT NOT NULL);
CREATE TABLE IF NOT EXISTS posts(id TEXT PRIMARY KEY, job TEXT NOT NULL);
''')
row = db.execute('SELECT value FROM settings WHERE id=1').fetchone()
self.run = json.loads(row[0]) if row else dict(running=False, epoch=0, config={}, batch=None, active=None, owner=None, switch=None, reason='请配置任务后开始')
self.run.update(running=False, epoch=self.run['epoch'] + 1, reason='服务已启动,请明确点击开始/继续;未决任务不会重发')
self._persist(db)
self.clients = {}
if ledger and Path(ledger).is_file():
self.import_history(json.loads(Path(ledger).read_text()).get('records', []))
@contextmanager
def db(self):
connection = sqlite3.connect(self.path, timeout=20)
try:
with connection:
yield connection
finally:
connection.close()
def _persist(self, db):
db.execute('INSERT OR REPLACE INTO settings VALUES(1,?)', (json.dumps(self.run, ensure_ascii=False),))
def _job(self, db, id):
row = db.execute('SELECT value FROM jobs WHERE id=?', (id,)).fetchone()
if not row:
raise ValueError('任务不存在')
return json.loads(row[0])
def _save(self, db, job):
job['updated_at'] = time.time()
db.execute('INSERT OR REPLACE INTO jobs VALUES(?,?,?)', (job['id'], job['article_key'], json.dumps(job, ensure_ascii=False)))
def import_history(self, records):
with self.lock, self.db() as db:
for record in records:
digest = record.get('markdown_sha256')
if digest and record.get('block_repost'):
db.execute('INSERT OR IGNORE INTO ledger VALUES(?,?)', ('md:' + digest, json.dumps(record, ensure_ascii=False)))
# Add content identity where the original historical file is still present.
try:
article = inspect_article(Path(record['markdown_path']).parent)
db.execute('INSERT OR IGNORE INTO ledger VALUES(?,?)', ('article:' + article['article_key'], json.dumps(record, ensure_ascii=False)))
except (OSError, ValueError, KeyError):
pass
def enqueue(self, directory):
article = inspect_article(directory)
article.pop('package')
keys = ['article:' + article['article_key'], 'md:' + article['markdown_sha256']] + ['source:' + str(k) for k in article['source_keys']]
with self.lock, self.db() as db:
if any(db.execute('SELECT 1 FROM ledger WHERE key=?', (k,)).fetchone() for k in keys):
return '已受理,跳过'
if db.execute('SELECT 1 FROM jobs WHERE article_key=?', (article['article_key'],)).fetchone():
return '已在队列,跳过'
# Source URL identities prevent different renderings of one downloaded article re-entering.
for row in db.execute('SELECT value FROM jobs'):
if set(article['source_keys']) & set(json.loads(row[0]).get('source_keys', [])):
return '相同来源已在队列,跳过'
article.update(id=secrets.token_hex(12), state='queued', created_at=time.time(), error='', post_url='')
self._save(db, article)
return '已入队'
def state(self):
with self.lock, self.db() as db:
jobs = [json.loads(r[0]) for r in db.execute('SELECT value FROM jobs ORDER BY rowid')]
return dict(run=json.loads(json.dumps(self.run)), jobs=jobs,
clients=[v for v in self.clients.values() if time.time() - v['seen'] < 20],
history_count=db.execute('SELECT count(*) FROM ledger').fetchone()[0])
def configure(self, config):
fid, minimum, maximum = config.get('fid'), config.get('minimum'), config.get('maximum')
interval = config.get('interval')
if any(type(x) is not int for x in (fid, minimum, maximum, interval)) or not (fid > 0 and 1 <= minimum <= maximum <= 1000 and 0 <= interval <= 86400):
raise ValueError('请填写有效版块 ID、篇数范围(1~1000)和间隔秒数')
if config.get('history_checked') is not True:
raise ValueError('请先核对待处理文章与历史发布记录,并勾选确认')
with self.lock, self.db() as db:
if self.run['active'] or self.run['switch'] or self.run['running']:
raise ValueError('当前流程未结束,不能更换配置')
self.run.update(config={k: config[k] for k in ('fid', 'minimum', 'maximum', 'interval', 'history_checked')}, batch=None)
self._persist(db)
def control(self, action):
with self.lock, self.db() as db:
if action == 'start':
if not self.run['config']:
raise ValueError('请先保存配置')
self.run.update(running=True, reason='等待浏览器连接')
else:
self.run.update(running=False, reason='已终止' if action == 'stop' else '已暂停')
self.run['epoch'] += 1
self._persist(db)
return dict(self.run)
def _batch(self, account):
cfg = self.run['config']
return dict(account=account, target=cfg['minimum'] + secrets.randbelow(cfg['maximum'] - cfg['minimum'] + 1), count=0)
def _check_owner(self, owner):
if self.run['owner'] != owner:
raise ValueError('任务由另一个浏览器标签持有')
def heartbeat(self, data):
owner = str(data.get('owner', ''))[:100]
if not owner:
raise ValueError('缺少执行标签标识')
with self.lock:
self.clients[owner] = dict(owner=owner, account=data.get('account'), url=str(data.get('url', ''))[:1000],
extension=data.get('extension'), seen=time.time())
return self.state()['run']
def next(self, data):
owner, account = data.get('owner'), data.get('account')
with self.lock, self.db() as db:
if not self.run['running']:
return dict(action='idle', run=self.run)
if not account or not str(account.get('uid', '')).isdigit() or not account.get('name'):
return dict(action='login', run=self.run)
if self.run['owner'] not in (None, owner):
return dict(action='other_tab', run=self.run)
self.run['owner'] = owner
if self.run['switch']:
switch = self.run['switch']
if account['uid'] != switch['from'] and data.get('extension', {}).get('idle') is True:
self.run.update(batch=self._batch(account), switch=None)
self._persist(db)
else:
return dict(action='switch' if switch['state'] == 'pending' else 'switch_wait', run=self.run)
if self.run['batch'] is None:
self.run['batch'] = self._batch(account)
if self.run['batch']['account']['uid'] != account['uid']:
self.run.update(running=False, reason='检测到账号变化,请先核对当前任务')
self._persist(db)
return dict(action='idle', run=self.run)
if self.run['active']:
job = self._job(db, self.run['active'])
return dict(action='resume', job=job, run=self.run)
rows = [json.loads(r[0]) for r in db.execute('SELECT value FROM jobs ORDER BY rowid')]
job = next((j for j in rows if j['state'] == 'queued'), None)
if not job:
self.run.update(running=False, reason='当前队列已处理完;新文章入队后请点击继续')
self._persist(db)
return dict(action='idle', run=self.run)
if time.time() < self.run.get('not_before', 0):
return dict(action='interval', run=self.run)
if self.run['batch']['count'] >= self.run['batch']['target']:
self.run['switch'] = dict(state='pending', **{'from': account['uid']}, created_at=time.time())
self._persist(db)
return dict(action='switch', run=self.run)
job.update(state='preparing', account=account, fid=self.run['config']['fid'], error='')
self.run['active'] = job['id']
self._save(db, job)
self._persist(db)
return dict(action='prepare', job=job, run=self.run)
def current(self, data):
with self.lock, self.db() as db:
self._check_owner(data['owner'])
return {'job': self._job(db, self.run['active']) if self.run['active'] else None, 'run': self.run}
def switch_begin(self, data):
with self.lock, self.db() as db:
self._check_owner(data['owner'])
if not self.run['running'] or not self.run['switch'] or self.run['switch']['state'] != 'pending':
raise ValueError('换号状态已改变,禁止重复点击')
self.run['switch']['state'] = 'clicked'
self._persist(db)
return {'ok': True}
def package(self, data):
with self.lock, self.db() as db:
self._check_owner(data['owner'])
if not self.run['running'] or self.run['active'] != data['id']:
raise ValueError('流程已停止或任务不匹配')
job = self._job(db, data['id'])
if job['state'] != 'preparing':
raise ValueError('当前任务不允许重新导入,请先核对已上传内容')
article = inspect_article(job['directory'])
if article['markdown_sha256'] != job['markdown_sha256'] or article['article_key'] != job['article_key']:
raise ValueError('入队后原文章已改变,请移出并重新扫描')
return article['package']
def event(self, data):
with self.lock, self.db() as db:
self._check_owner(data['owner'])
job = self._job(db, data['id'])
if self.run['active'] != job['id']:
raise ValueError('不是当前任务')
target = data['state']
transitions = {'preparing': {'importing', 'needs_attention'}, 'importing': {'formatting', 'needs_attention'},
'formatting': {'awaiting_user_review', 'needs_attention'},
'awaiting_user_review': {'user_submitting', 'needs_attention'},
'user_submitting': {'submit_unknown', 'needs_attention'}, 'submit_unknown': {'needs_attention'}}
if target not in transitions.get(job['state'], set()):
raise ValueError('任务状态不允许此操作:' + job['state'] + ' → ' + target)
if target in ('importing', 'formatting', 'awaiting_user_review') and (not self.run['running'] or data.get('epoch') != self.run['epoch']):
raise ValueError('流程已暂停/终止,过期回调不能继续')
job.update(state=target, error=str(data.get('error', ''))[:1000])
if target == 'user_submitting':
job.update(submitted_at=time.time(), snapshot=data.get('snapshot', {}), attempt=secrets.token_hex(8))
if target == 'awaiting_user_review':
job['prepared'] = data.get('snapshot', {})
if target == 'needs_attention':
self.run.update(running=False, reason=job['error'] or '当前任务需要处理')
self._persist(db)
self._save(db, job)
return {'ok': True}
def accept(self, data):
with self.lock, self.db() as db:
self._check_owner(data['owner'])
job = self._job(db, data['id'])
if job['state'] in ACCEPTED:
return {'ok': True, 'duplicate': True}
if self.run['active'] != job['id'] or job['state'] not in SUBMITTED:
raise ValueError('只有已观察到人工提交的任务才能登记结果')
url = urlsplit(str(data.get('post_url', '')))
import re
match = re.fullmatch(r'/thread/(\d+)\.html', url.path)
if url.scheme != 'https' or url.netloc not in ('www.dww11.com', 'dww11.com') or not match:
raise ValueError('未识别到帖子详情页')
if data.get('author_uid') != job['account']['uid'] or data.get('fid') != job['fid']:
raise ValueError('结果页作者或版块与任务不符,请人工核对')
if data.get('title') != job.get('snapshot', {}).get('title'):
raise ValueError('结果页标题与人工提交快照不一致')
if data.get('state') not in ACCEPTED:
raise ValueError('需要明确的公开或审核中状态')
if not data.get('evidence'):
raise ValueError('缺少页面结果证据')
post_id = match[1]
existing = db.execute('SELECT job FROM posts WHERE id=?', (post_id,)).fetchone()
if existing and existing[0] != job['id']:
raise ValueError('帖子已绑定其他任务,不能重复计数')
db.execute('INSERT OR IGNORE INTO posts VALUES(?,?)', (post_id, job['id']))
job.update(state=data['state'], post_id=post_id, post_url=url.geturl(), result=data['evidence'])
keys = ['article:' + job['article_key'], 'md:' + job['markdown_sha256']] + ['source:' + str(k) for k in job['source_keys']]
for key in keys:
db.execute('INSERT OR REPLACE INTO ledger VALUES(?,?)', (key, json.dumps(job, ensure_ascii=False)))
self.run['batch']['count'] += 1
self.run.update(active=None, not_before=time.time() + self.run['config']['interval'])
self._save(db, job)
self._persist(db)
return {'ok': True}
def resolve(self, data):
"""Explicit local UI resolution, never an automatic retry of an unknown submission."""
with self.lock, self.db() as db:
job = self._job(db, data['id'])
if job['state'] in ACCEPTED:
raise ValueError('已受理文章不能重置')
if self.run['running']:
raise ValueError('请先暂停流程')
if data.get('confirmed_not_accepted') is not True:
raise ValueError('必须先人工核对网站未受理,并清理编辑器/草稿')
job.update(state='queued' if data.get('retry') else 'cancelled', error='', snapshot={})
if self.run['active'] == job['id']:
self.run.update(active=None, owner=None)
self._save(db, job)
self._persist(db)
return {'ok': True}
def takeover(self, data):
with self.lock, self.db() as db:
if self.run['running']:
raise ValueError('请先在控制台暂停流程,再接管')
self.run['owner'] = data['owner']
self._persist(db)
return {'ok': True}
+12
View File
@@ -0,0 +1,12 @@
# 测试与归档
- `history/`:从 auto_news_upload 根目录移入的历史截图和演示任务 JSON,保留原内容。不是自动测试,不能用作本次测试结论。
- `fixtures/article/`:自造 Markdown 和三张 1×1 PNG,其中两张内容相同,用于验证上传去重及插图顺序。
- `test_queue.py`:持久队列、去重、重启、终止、未知提交和按篇数换号。
- `test_controller.py`:本地接口权限、模拟下载到队列、静态页面路由。
- `test_download_core.py`:复制的下载器原有离线测试,针对合并项目中的下载器副本运行。
- `importer.test.cjs`:导入器回归与自动模式的停止/冲突保护。
- `browser.test.cjs`:本地页面、移动布局与模拟编辑器;从不请求真实论坛。
- `artifacts/`:本轮页面截图,属于生成产物。
原始受理去重记录继续保存在 `../发布去重记录.json`,服务启动时自动导入;没有作为测试残留移走或删除。
+13
View File
@@ -0,0 +1,13 @@
'use strict';
const {test}=require('node:test');const assert=require('node:assert/strict');const fs=require('node:fs');const path=require('node:path');const vm=require('node:vm');
test('终止后较早的 heartbeat 响应不能恢复流程',async()=>{
const handlers=new Map(),requests=[],responses=[],values={pairToken:'test-token',pairPort:8770};
class Event{constructor(type,options={}){this.type=type;this.detail=options.detail;}}
const doc={readyState:'loading',addEventListener(type,fn){handlers.set(type,fn);},dispatchEvent(event){if(event.type==='dww11:local-response')responses.push(JSON.parse(event.detail));else handlers.get(event.type)?.(event);}};
vm.runInNewContext(fs.readFileSync(path.join(__dirname,'../browser/local-bridge.user.js'),'utf8'),{document:doc,CustomEvent:Event,GM_getValue:(k,d)=>values[k]??d,GM_setValue:(k,v)=>values[k]=v,GM_addValueChangeListener:()=>{},GM_xmlhttpRequest:r=>requests.push(r)});
const call=(id,route)=>handlers.get('dww11:local-request')(new Event('',{detail:JSON.stringify({id,route})}));
const heartbeat=call('old','heartbeat');const stop=call('stop','stop');
requests[0].onload({status:200,responseText:JSON.stringify({running:true,epoch:1})});await heartbeat;
assert.equal(responses.find(r=>r.id==='old').value.running,false);assert.equal(values.halted,true);
requests[1].onload({status:200,responseText:JSON.stringify({running:false,epoch:2})});await stop;
});
+91
View File
@@ -0,0 +1,91 @@
'use strict';
const {chromium}=require('playwright');
const fs=require('node:fs');
const path=require('node:path');
const assert=require('node:assert/strict');
const ROOT=path.resolve(__dirname,'..');
const output=path.join(__dirname,'artifacts');fs.mkdirSync(output,{recursive:true});
(async()=>{
const browser=await chromium.launch({headless:true,executablePath:process.env.CHROME_PATH||'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'});
try{
const page=await browser.newPage({viewport:{width:1440,height:1100}}),errors=[];
page.on('pageerror',error=>errors.push(error.message));
await page.route('**/*',route=>new URL(route.request().url()).hostname==='127.0.0.1'?route.continue():route.abort());
await page.goto('http://127.0.0.1:8771/');
await page.getByText('原创导入测试稿',{exact:true}).waitFor();
await page.screenshot({path:path.join(output,'desktop.png'),fullPage:true});
await page.locator('#fid').fill('125');await page.locator('#minimum').fill('1');await page.locator('#maximum').fill('2');await page.locator('#interval').fill('10');await page.locator('#history-checked').check();
await page.getByRole('button',{name:'保存处理规则'}).click();await page.getByText('处理规则已保存',{exact:true}).waitFor();
await page.locator('#start').click();await page.getByText('流程运行中',{exact:true}).waitFor();
await page.locator('#stop').click();await page.getByText('流程已暂停',{exact:true}).waitFor();
await page.setViewportSize({width:390,height:844});await page.screenshot({path:path.join(output,'mobile.png'),fullPage:true});
assert.equal(await page.evaluate(()=>document.documentElement.scrollWidth>innerWidth),false,'mobile overflow');
assert.deepEqual(errors,[]);console.log('PASS local UI: configuration, start/stop, desktop/mobile layout');
await page.close();
const mock=await browser.newPage();
await mock.route('**/*',route=>route.fulfill({contentType:'text/html; charset=utf-8',body:'<!doctype html><a id="mumucms_username" href="/home.php?mod=space&uid=123">测试账号</a><input id="subject"><iframe id="e_iframe"></iframe><button id="postsubmit">发表帖子</button>'}));
await mock.goto('https://www.dww11.com/forum.php?mod=post&action=newthread&fid=125');
await mock.evaluate(()=>{
window.wysiwyg=1;window.events=[];window.clicks=0;window.formats=0;
window.job={id:'fixture',state:'preparing',title:'测试标题',source:'测试来源',fid:125,account:{uid:'123',name:'测试账号'}};
window.flow={running:true,epoch:1,active:'fixture',config:{fid:125}};
document.querySelector('#postsubmit').addEventListener('click',()=>window.clicks++);
window.DWW11Importer={stagePackage:async()=>({warnings:[]}),importPrepared:async()=>{
document.querySelector('#subject').value='测试标题';
document.querySelector('#e_iframe').contentDocument.body.innerHTML='<p>正文第一段</p><img id="attachimg_1" src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7"><p>正文第二段</p><p>来源:测试来源,如有侵权,请联系删除。</p>';
return {images:{'a.gif':'1'}};
}};
document.addEventListener('dww11:local-request',event=>{
const {id,route,body}=JSON.parse(event.detail);let value={};
window.events.push({route,body});
if(route==='heartbeat'){window.flow.owner=body.owner;value=window.flow;}
if(route==='next')value={action:window.job.state==='preparing'?'prepare':'resume',job:window.job,run:window.flow};
if(route==='current')value={job:window.job,run:window.flow};
if(route==='event'){window.job.state=body.state;value={ok:true};}
if(route==='stop'){window.flow.running=false;value=window.flow;}
document.dispatchEvent(new CustomEvent('dww11:local-response',{detail:JSON.stringify({id,value})}));
});
});
await mock.addScriptTag({path:path.join(ROOT,'browser/formatter.js')});
await mock.addScriptTag({path:path.join(ROOT,'browser/runner.js')});
await mock.waitForFunction(()=>window.job.state==='awaiting_user_review');
await mock.waitForTimeout(2400);
assert.equal(await mock.evaluate(()=>window.clicks),0,'runner must never publish');
assert.equal(await mock.evaluate(()=>document.querySelector('#e_iframe').contentDocument.body.querySelectorAll('[data-dww11-wx-source]').length),1);
const before=await mock.evaluate(()=>document.querySelector('#e_iframe').contentDocument.body.innerHTML);
await mock.evaluate(()=>window.DWW11Formatter.format('测试来源'));
const after=await mock.evaluate(()=>document.querySelector('#e_iframe').contentDocument.body.innerHTML);
assert.equal(after,before,'repeated formatting should be idempotent');
await mock.locator('#postsubmit').click();
await mock.waitForFunction(()=>window.job.state==='user_submitting');
await mock.evaluate(()=>history.pushState({},'', '/thread/999.html'));
await mock.waitForFunction(()=>window.job.state==='submit_unknown');
assert.equal(await mock.evaluate(()=>window.flow.running),false);
assert.equal(await mock.evaluate(()=>window.events.some(e=>e.route==='accept')),false,'unknown page must not count accepted');
assert.equal(await mock.evaluate(()=>window.clicks),1);
console.log('PASS browser fixture: import/format, source uniqueness, repeated formatting, manual-only publish, unknown result stops');
await mock.close();
const ext=await browser.newPage();
await ext.route('**/*',route=>route.fulfill({contentType:'text/html; charset=utf-8',body:'<!doctype html><div class="topwp"><a href="#">立即登录</a></div>'}));
await ext.goto('https://www.dww11.com/');
await ext.evaluate(()=>{
window.routes=[];window.chrome.runtime={sendMessage:async message=>{
window.routes.push(message.route);
if(message.route==='/hello')return {ok:true,data:{total:1,active:1,round:1,remaining:1,flow:null}};
if(message.route==='/start')return new Promise(resolve=>window.finishStart=()=>resolve({ok:true,data:{flow:{id:'test-run',phase:'logout',paused:false}}}));
if(message.route==='/suspend')return {ok:true,data:{}};
throw new Error('Unexpected extension operation: '+message.route);
}};
});
await ext.addScriptTag({path:path.join(ROOT,'../dww11-account-switch-browser-extension/content.js')});
await ext.waitForFunction(()=>JSON.parse(document.querySelector('#dww11-extension-panel').dataset.autoNewsState).idle);
await ext.locator('#dww11-extension-next').click();
await ext.waitForFunction(()=>!!window.finishStart);
await ext.evaluate(()=>{document.dispatchEvent(new CustomEvent('dww11:auto-stop'));window.finishStart();});
await ext.waitForFunction(()=>window.routes.includes('/suspend'));
assert.equal(await ext.evaluate(()=>JSON.parse(document.querySelector('#dww11-extension-panel').dataset.autoNewsState).paused),true);
assert.deepEqual(await ext.evaluate(()=>window.routes),['/hello','/start','/suspend']);
console.log('PASS extension bridge: stop during pending start prevents login and persists pause');
await ext.close();
}finally{await browser.close();}
})().catch(error=>{console.error(error);process.exitCode=1;});
Binary file not shown.

After

Width:  |  Height:  |  Size: 69 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 69 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 69 B

+15
View File
@@ -0,0 +1,15 @@
# 原创导入测试稿
这是项目自有的离线测试稿,用于验证正文、段落和图片顺序。
![](image001.png)
第二段保留原文标点,以及普通文字。
![](image002.png)
最后一段说明图片上传不等于发表文章。
![](image003.png)
来源:项目测试素材,如有侵权,请联系删除。
+172
View File
@@ -0,0 +1,172 @@
// Offline verification only. No browser account, uploads or publication requests.
'use strict';
const { test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const vm = require('node:vm');
const { webcrypto, createHash } = require('node:crypto');
const { execFileSync } = require('node:child_process');
const core = require('../browser/importer.js');
const directory = path.join(__dirname, 'fixtures/article');
const mdName = '原创导入测试稿.md';
function articleFiles() {
return fs.readdirSync(directory).map(name => {
const blob = new Blob([fs.readFileSync(path.join(directory, name))], { type: name.endsWith('.png') ? 'image/png' : 'text/markdown' });
blob.name = name; blob.webkitRelativePath = '文章/' + name; return blob;
});
}
test('测试文章:标题、3 张配图、插入顺序、来源与段落完整', async () => {
const article = await core.prepareFiles(articleFiles());
assert.equal(article.title, mdName.slice(0, -3));
assert.deepEqual([...article.images.keys()], ['文章/image001.png', '文章/image002.png', '文章/image003.png']);
const output = article.render(new Map([...article.images.keys()].map((p, i) => [p, String(101 + i)])));
assert.deepEqual([...output.matchAll(/\[attachimg\](\d+)\[\/attachimg\]/g)].map(m => m[1]), ['101', '102', '103']);
assert.equal(output.includes('# 原创导入测试稿'), false);
const md = fs.readFileSync(path.join(directory, mdName), 'utf8');
const originalBody = md.split('\n').slice(1).join('\n').trim().replace(/!\[\]\(image00(\d)\.png\)/g, (_, n) => `[attachimg]${100 + Number(n)}[/attachimg]`);
assert.equal(output, originalBody);
});
test('缺图、远程图片、超出目录、超大图片与超长标题在上传前被拒绝', () => {
assert.throws(() => core.compile('# 标题\n\n![](missing.jpg)', 'a.md', new Map()), /找不到配图/);
assert.throws(() => core.compile('# 标题\n\n![](https://x/a.jpg)', 'a.md', new Map()), /目录内/);
assert.throws(() => core.imagePath('a.md', '../secret.jpg'), /超出/);
assert.throws(() => core.imagePath('a.md', '%2fetc/passwd'), /相对路径/);
assert.throws(() => core.compile('# ' + '长'.repeat(81) + '\n正文', 'a.md', new Map()), /80/);
assert.throws(() => core.compile('# 标题\n\n![](big.png)', 'a.md', new Map([['big.png', {size:20*1024*1024}]])), /20 MB/);
});
test('子目录、Unicode 规范化、空格编码、括号、重复图片和格式转换', () => {
const fileMap = new Map([['目录/图 (1).jpg', new Blob(['image'])]]);
const article = core.compile('# 标题\n\n**粗体**和*斜体*\n\n![](图%20(1).jpg)\n![](<图 (1).jpg> "说明")\n\n[链接](https://example.com/a)\n\n- 一\n- 二\n\n> 引用', '目录/a.md', fileMap);
assert.equal(article.images.size, 1);
assert.match(article.render(new Map([['目录/图 (1).jpg', '100']])), /\[b\]粗体\[\/b\]/);
assert.equal(core.normalizePath('cafe\u0301/a.jpg'), 'café/a.jpg');
assert.equal(core.readLink("![](child's.jpg)", 0).target, "child's.jpg");
assert.throws(() => core.compile('# 标题\n![][a]\n[a]: x.jpg', 'a.md', fileMap), /引用式/);
});
test('多个 Markdown 必须选择;无 Markdown 不接受', async () => {
const files = articleFiles();
const extra = new Blob(['# 另一篇\n正文']); extra.name = 'b.md'; extra.webkitRelativePath = '文章/b.md';
const result = await core.prepareFiles([...files, extra]);
assert.equal(result.candidates.length, 2);
assert.equal((await core.prepareFiles([...files, extra], '文章/b.md')).title, '另一篇');
await assert.rejects(core.prepareFiles(files.filter(f => !f.name.endsWith('.md'))), /没有/);
});
test('下载程序对接:Python 文章包与目录直接导入完全一致', async () => {
const data = JSON.parse(execFileSync('python3', [path.join(__dirname, '../modules/package.py'), directory], {encoding:'utf8'}));
const packaged = await core.preparePackage(data);
const direct = await core.prepareFiles(articleFiles());
assert.equal(packaged.title, direct.title);
assert.equal(packaged.body, direct.body);
assert.equal(packaged.images.size, 3);
for (const [p, blob] of packaged.images) assert.equal(blob.size, fs.statSync(path.join(directory, p)).size);
});
test('服务器返回错误、登录 HTML 或混入脚本时不猜测附件编号', () => {
assert.equal(core.parseAid(' 123456\n'), '123456');
for (const value of ['-1', '0', '<html>123</html>', '<script>123</script>', '123error', '{"aid":123}']) assert.throws(() => core.parseAid(value));
});
// Small DOM harness exercises the public API and site adapter without installing browser software.
function harness({ failUpload = false, loseImages = false, existing = false } = {}) {
const ids = new Map(), callbacks = new Map(), requests = [];
class Element {
constructor(id='') { this.id=id; this.style={}; this.value=''; this.disabled=false; this.checked=false; this.hidden=false; this.listeners=new Map(); this.children=[]; }
attachShadow() { return this; }
set innerHTML(value) { this._html=value; for (const m of value.matchAll(/\bid="([^"]+)"/g)) ids.set(m[1], new Element(m[1])); }
get innerHTML() { return this._html || ''; }
getElementById(id) { return ids.get(id); }
appendChild(el) { this.children.push(el); return el; }
replaceChildren(...els) { this.children=els; }
addEventListener(name, fn) { this.listeners.set(name, fn); }
removeEventListener(name) { this.listeners.delete(name); }
dispatchEvent() {}
click() { this.onclick?.(); }
querySelectorAll() { return []; }
}
for (const id of ['subject', 'e_textarea', 'e_switchercheck', 'postform']) ids.set(id, new Element(id));
ids.get('subject').value=existing ? '原标题' : '';
ids.get('e_textarea').value=existing ? '原正文' : '';
const doc = { body:new Element(), createElement: () => new Element(), getElementById:id=>ids.get(id),
querySelector: s=>ids.get(s.replace(/^#/,'')), querySelectorAll:()=>[] };
const win = { document:doc, location:{search:'?mod=post&action=newthread&fid=95',href:'https://www.dww11.com/forum.php?mod=post&action=newthread&fid=95',origin:'https://www.dww11.com'},
wysiwyg:0, allowbbcode:1, allowimgcode:1, crypto:webcrypto,
imgUpload:{settings:{upload_url:'https://www.dww11.com/misc.php?mod=swfupload&operation=upload&fid=95',post_params:{uid:'fake-user',hash:'fake-hash',type:'image'},file_size_limit:20480}},
confirm:()=>true, dispatchEvent:()=>{}, addEventListener:()=>{}, editdoc:{body:new Element()} };
win.getEditorContents=()=>win.wysiwyg ? win.editdoc.body.innerHTML : ids.get('e_textarea').value;
win.switchEditor=mode=>{
if (mode===win.wysiwyg) return;
if (mode===1) win.editdoc.body.innerHTML=ids.get('e_textarea').value.replace(/\[attachimg\](\d+)\[\/attachimg\]/g, (_,aid)=>loseImages ? '' : `<img aid="attachimg_${aid}">`);
else ids.get('e_textarea').value=win.editdoc.body.innerHTML;
win.wysiwyg=mode;
};
win.updateImageList=(_action,aids)=>{ for (const id of aids.split('|')) ids.set('image_'+id,new Element()); };
win.fetch=async (url,options)=>{
requests.push({url,options});
return {ok:true,text:async()=>failUpload ? '-1' : String(1000+requests.length)};
};
const context = {window:win,URL,URLSearchParams,Blob,FormData,AbortController,Uint8Array,Option:Element,Event:class {},CustomEvent:class {},
MutationObserver:class {observe(){} disconnect(){}},atob,setTimeout,clearTimeout,console};
vm.runInNewContext(fs.readFileSync(path.join(__dirname, '../browser/importer.js'),'utf8'), context);
return {api:win.DWW11Importer, win, ids, requests};
}
test('完整导入:暂存零请求、相同图片去重、3处附件映射、富文本结果、重试复用、没有发布请求', async () => {
const h=harness();
await h.api.stageFiles(articleFiles());
assert.equal(h.requests.length,0);
const result=await h.api.importPrepared();
assert.equal(Object.keys(result.images).length,3);
const uniqueImages=new Set(fs.readdirSync(directory).filter(p=>p.endsWith('.png')).map(p=>createHash('sha256').update(fs.readFileSync(path.join(directory,p))).digest('hex'))).size;
assert.equal(h.requests.length,uniqueImages);
assert.equal(h.ids.get('subject').value,mdName.slice(0,-3));
assert.match(h.win.getEditorContents(),/attachimg_1001/);
assert.equal(h.api.getState().busy,false);
assert.equal(h.api.getBackup().content,'');
await h.api.importPrepared();
assert.equal(h.requests.length,uniqueImages);
for (const req of h.requests) {
assert.equal(new URL(req.url).pathname,'/misc.php');
assert.equal(req.options.body.get('type'),'image');
assert.ok(req.options.body.get('Filedata') instanceof Blob);
}
});
test('上传失败保留原文;富文本丢图片时回滚;取消覆盖不上传', async () => {
for (const options of [{failUpload:true,existing:true},{loseImages:true,existing:true}]) {
const h=harness(options);
await h.api.stageFiles(articleFiles());
await assert.rejects(h.api.importPrepared());
assert.equal(h.ids.get('subject').value,'原标题');
assert.equal(h.win.getEditorContents(),'原正文');
assert.equal(h.api.getState().busy,false);
}
const h=harness({existing:true}); h.win.confirm=()=>false;
await h.api.stageFiles(articleFiles());
assert.equal((await h.api.importPrepared()).cancelled,true);
assert.equal(h.requests.length,0);
});
// New integration guards must not prompt or overwrite a user's draft.
test('自动导入:非空编辑器停止,不弹窗、不上传', async () => {
const h=harness({existing:true});h.win.confirm=()=>{throw new Error('unexpected prompt');};
await h.api.stageFiles(articleFiles());
await assert.rejects(h.api.importPrepared({automatic:true}),/编辑器已有内容/);
assert.equal(h.requests.length,0);assert.equal(h.ids.get('subject').value,'原标题');
});
test('自动导入:预先终止不上传,不填写', async () => {
const h=harness();await h.api.stageFiles(articleFiles());const c=new AbortController();c.abort(new Error('stopped'));
await assert.rejects(h.api.importPrepared({automatic:true,signal:c.signal}),/stopped/);
assert.equal(h.requests.length,0);assert.equal(h.ids.get('subject').value,'');
});
test('自动导入:上传中终止不会填写正文或继续下一张', async () => {
const h=harness();await h.api.stageFiles(articleFiles());const c=new AbortController();
const original=h.win.fetch;h.win.fetch=async (...args)=>{const result=await original(...args);c.abort(new Error('stopped'));return result;};
await assert.rejects(h.api.importPrepared({automatic:true,signal:c.signal}),/stopped/);
assert.equal(h.requests.length,1);assert.equal(h.ids.get('subject').value,'');
});
+19
View File
@@ -0,0 +1,19 @@
"""Temporary local UI fixture; never contacts the news API or forum."""
from pathlib import Path
import sys
import tempfile
from http.server import ThreadingHTTPServer
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
from app import Controller, make_handler
with tempfile.TemporaryDirectory() as temp:
controller = Controller(Path(temp) / 'data', Path(temp) / 'downloads', ledger=None)
controller.store.enqueue(ROOT / 'test/fixtures/article')
server = ThreadingHTTPServer(('127.0.0.1', 8771), make_handler(controller))
print('UI fixture listening on 8771', flush=True)
try:
server.serve_forever()
except KeyboardInterrupt:
pass
finally:
server.server_close()
+75
View File
@@ -0,0 +1,75 @@
import json
from pathlib import Path
import sys
import tempfile
import threading
import unittest
from http.server import ThreadingHTTPServer
from urllib.request import Request, urlopen
from urllib.error import HTTPError
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
from app import Controller, make_handler
class ControllerTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.root = Path(self.tmp.name)
self.controller = Controller(self.root / 'data', self.root / 'downloads', ledger=None)
self.server = ThreadingHTTPServer(('127.0.0.1', 0), make_handler(self.controller))
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
self.thread.start()
self.url = 'http://127.0.0.1:' + str(self.server.server_port)
def tearDown(self):
self.server.shutdown();self.server.server_close();self.thread.join();self.tmp.cleanup()
def request(self, path, body=None, token=None, origin=None, host=None):
headers = {}
if token: headers['X-Session-Token'] = token
if origin: headers['Origin'] = origin
if host: headers['Host'] = host
if body is not None: headers['Content-Type'] = 'application/json'
request = Request(self.url + path, data=None if body is None else json.dumps(body).encode(), headers=headers)
with urlopen(request, timeout=5) as response:
return response.status, response.read()
def test_auth_and_origin_boundaries(self):
for path, kwargs in [('/api/state', {}), ('/api/state', {'token': self.controller.bridge_token}),
('/api/state', {'token': self.controller.ui_token, 'origin': 'https://example.com'}),
('/bridge/heartbeat', {'token': self.controller.ui_token})]:
with self.assertRaises(HTTPError) as error:
self.request(path, {}, **kwargs)
self.assertEqual(error.exception.code, 403)
with self.assertRaises(HTTPError):
self.request('/api/session', host='attacker.example')
code, body = self.request('/api/state', {}, token=self.controller.ui_token)
self.assertEqual(code, 200)
self.assertNotIn(self.controller.bridge_token, body.decode())
def test_static_and_unknown_routes(self):
self.assertIn('新闻工作台', self.request('/')[1].decode())
self.assertEqual(self.request('/browser/auto-news.user.js')[0], 200)
with self.assertRaises(HTTPError):
self.request('/../store.py')
with self.assertRaises(HTTPError):
self.request('/bridge/scan', {}, token=self.controller.bridge_token)
def test_download_to_queue_without_network(self):
self.controller.manager.requester = lambda *_: {'code': 1, 'data': {'list': [dict(title='模拟新闻', src='模拟来源', content='<p>模拟正文</p>', url='https://example.com/news')]}}
self.controller.ui('download', dict(key='fake-key', channel='模拟频道', target=1))
self.controller.manager.thread.join(5)
state = self.controller.ui('state', {})
self.assertEqual(state['download']['saved'], 1)
self.assertEqual(len(state['jobs']), 1)
self.assertEqual(state['jobs'][0]['state'], 'queued')
self.assertTrue(state['jobs'][0]['source_keys'])
self.assertNotIn('fake-key', json.dumps(state))
self.controller.scan(self.controller.manager.root)
self.assertEqual(len(self.controller.store.state()['jobs']), 1)
if __name__ == '__main__':
unittest.main()
+165
View File
@@ -0,0 +1,165 @@
import base64
import json
from pathlib import Path
import sys
import tempfile
import threading
import unittest
from http.server import ThreadingHTTPServer
import requests
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from app import downloader
Downloader, make_handler, fingerprints = downloader.Downloader, downloader.make_handler, downloader.fingerprints
from article_writer import save_article
def article(i):
return dict(title=f'新闻{i}', content=f'<p>第{i}篇新闻内容</p>', url=f'https://example.com/{i}', src='测试来源', time='2026-10-31')
class DownloadTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.root = Path(self.temp.name)
def tearDown(self):
self.temp.cleanup()
def run_job(self, manager, target):
manager.start('test-key', '头条', target)
manager.thread.join(10)
self.assertFalse(manager.thread.is_alive())
return manager.snapshot()
def test_format_and_local_image(self):
data = article(1)
image = 'data:image/png;base64,' + base64.b64encode(b'\x89PNG\r\n\x1a\nexample').decode()
data['content'] += f'<img src="{image}"><img src="{image}">'
result = save_article(data, self.root)
body = Path(result.markdown).read_text()
self.assertTrue(body.startswith('# 新闻1\n\n第1篇新闻内容'))
self.assertNotIn('发布时间:2026-10-31', body)
self.assertNotIn('\n---\n', body)
self.assertTrue(body.endswith('来源:测试来源,如有侵权,请联系删除。\n'))
self.assertEqual(len(result.images), 1)
self.assertEqual(body.count('image001.png'), 2)
self.assertNotIn('data:image', body)
second = save_article(data, self.root)
self.assertEqual(Path(second.markdown).name, '新闻1 (2).md')
def test_bad_image_cleans_partial(self):
data = article(1)
data['content'] += '<img src="data:image/png;base64,aW52YWxpZA==">'
with self.assertRaises(Exception):
save_article(data, self.root)
self.assertEqual(list(self.root.iterdir()), [])
def test_duplicate_then_fill_and_restart(self):
items = [article(1), article(1), article(2), article(3)]
calls = []
def request(key, endpoint, params):
calls.append(params.copy())
return {'code': 1, 'data': {'list': items[params['start']:params['start']+params['num']]}}
manager = Downloader(self.root, request)
state = self.run_job(manager, 3)
self.assertEqual((state['status'], state['saved'], state['skipped']), ('completed', 3, 1))
self.assertEqual([c['start'] for c in calls], [0, 3])
restarted = Downloader(self.root, request)
state = self.run_job(restarted, 3)
self.assertEqual((state['status'], state['saved'], state['skipped']), ('partial', 0, 4))
self.assertEqual(len(list(self.root.glob('*/*.md'))), 3)
def test_offset_limit_and_partial_progress(self):
offsets = []
def request(key, endpoint, params):
offsets.append(params['start'])
return {'code': 1, 'data': {'list': [article(i) for i in range(params['start'], params['start']+params['num'])]}}
state = self.run_job(Downloader(self.root, request), 500)
self.assertEqual(state['status'], 'partial')
self.assertEqual(state['saved'], 440)
self.assertEqual(max(offsets), 400)
def test_error_and_key_redaction(self):
def request(*args):
return {'code': 10001, 'msg': 'invalid test-key'}
manager = Downloader(self.root, request)
state = self.run_job(manager, 10)
self.assertEqual(state['status'], 'error')
self.assertNotIn('test-key', json.dumps(state))
def test_stop_and_concurrent_start(self):
entered, release = threading.Event(), threading.Event()
def request(*args):
entered.set()
release.wait(2)
return {'code': 1, 'data': {'list': [article(1)]}}
manager = Downloader(self.root, request)
manager.start('key', '头条', 10)
self.assertTrue(entered.wait(2))
with self.assertRaises(ValueError):
manager.start('key', '头条', 10)
manager.stop.set()
release.set()
manager.thread.join(3)
self.assertEqual(manager.snapshot()['status'], 'cancelled')
self.assertEqual(manager.snapshot()['saved'], 0)
def test_failed_save_can_retry_next_job(self):
attempts = []
def writer(data, root):
attempts.append(1)
if len(attempts) == 1:
raise ValueError('模拟图片失败')
return save_article(data, root)
def request(key, endpoint, params):
return {'code': 1, 'data': {'list': [article(1)] if params['start'] == 0 else []}}
manager = Downloader(self.root, request, writer)
self.assertEqual(self.run_job(manager, 1)['failed'], 1)
self.assertEqual(self.run_job(manager, 1)['saved'], 1)
def test_fingerprint_content_and_tracking(self):
a, b = article(1), article(2)
b['content'] = a['content'].replace('<p>', '<div>').replace('</p>', '</div>')
self.assertTrue(set(fingerprints(a)) & set(fingerprints(b)))
b['content'] = '其他内容'
b['url'] = a['url'] + '?utm_source=abc#fragment'
self.assertTrue(set(fingerprints(a)) & set(fingerprints(b)))
def test_log_cap(self):
manager = Downloader(self.root)
for i in range(400):
manager.log(str(i))
self.assertEqual(len(manager.snapshot()['logs']), 300)
self.assertEqual(manager.snapshot()['logs'][0]['message'], '100')
def test_http_routes_and_access(self):
def request(key, endpoint, params=None):
return {'code': 1, 'data': {'list': ['头条', '科技']}}
manager = Downloader(self.root, request)
server = ThreadingHTTPServer(('127.0.0.1', 0), make_handler(manager, 'token'))
worker = threading.Thread(target=server.serve_forever, daemon=True)
worker.start()
url = f'http://127.0.0.1:{server.server_port}'
try:
self.assertIn('新闻下载器', requests.get(url).text)
self.assertEqual(requests.get(url+'/api/state').json()['status'], 'idle')
self.assertEqual(requests.post(url+'/api/channels', json={'key': 'test'}).status_code, 403)
self.assertEqual(requests.get(url, headers={'Host':'evil.example'}).status_code, 403)
headers = {'X-Session-Token': 'token'}
self.assertEqual(requests.post(url+'/api/channels', headers=headers, json={'key':'test'}).json()['channels'], ['头条','科技'])
invalid = requests.post(url+'/api/start', headers=headers, json={'key':'test', 'target':0,'channel':'头条'})
self.assertEqual(invalid.status_code, 400)
invalid = requests.post(url+'/api/debug', headers=headers, json={'key':'test','endpoint':'index','num':41,'start':0})
self.assertEqual(invalid.status_code, 400)
debug = requests.post(url+'/api/debug', headers=headers, json={'key':'test','endpoint':'channel'})
self.assertEqual(debug.status_code, 200)
self.assertNotIn('key=test', debug.text)
finally:
server.shutdown()
server.server_close()
worker.join()
if __name__ == '__main__':
unittest.main()
+155
View File
@@ -0,0 +1,155 @@
import hashlib
import json
from pathlib import Path
import shutil
import sys
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
from articles import inspect_article, short_title
from store import Store
class QueueTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.root = Path(self.tmp.name)
self.article = self.root / 'article'
self.article.mkdir()
self.md = self.article / 'a.md'
self.md.write_text('# 测试新闻\n\n正文内容。\n\n来源:测试来源,如有侵权,请联系删除。\n')
self.store = Store(self.root / 'queue.sqlite3')
self.store.configure(dict(fid=125, minimum=1, maximum=1, interval=0, history_checked=True))
self.owner = dict(owner='tab-a', account=dict(uid='123', name='测试账号'), extension={'idle': True})
def tearDown(self):
self.tmp.cleanup()
def begin(self):
self.store.enqueue(self.article)
self.store.control('start')
return self.store.next(self.owner)['job']
def transition(self, job, state, **extra):
return self.store.event(dict(owner='tab-a', id=job['id'], state=state, epoch=self.store.run['epoch'], **extra))
def submit(self, job):
for state in ('importing', 'formatting', 'awaiting_user_review'):
self.transition(job, state)
self.transition(job, 'user_submitting', snapshot={'title': job['title']})
def accept(self, job):
return self.store.accept(dict(owner='tab-a', id=job['id'], post_url='https://www.dww11.com/thread/12345.html',
author_uid='123', fid=125, title=job['title'], state='accepted_pending_moderation', evidence={'test': 'fixture only'}))
def test_stable_identity_and_original_preserved(self):
before = self.md.read_bytes()
item = inspect_article(self.article)
copy = self.root / 'renamed'
shutil.copytree(self.article, copy)
(copy / 'a.md').write_text(self.md.read_text().replace('测试新闻', '改标题'))
self.assertEqual(inspect_article(copy)['article_key'], item['article_key'])
self.assertEqual(self.md.read_bytes(), before)
self.assertEqual(item['source'], '测试来源')
def test_title_does_not_break_emoji(self):
short = short_title('😀' * 50)
self.assertTrue(short.endswith('...'))
self.assertLessEqual(len(short.encode('utf-16-le')) // 2, 80)
self.assertEqual(short_title('短标题'), '短标题')
def test_unsupported_images_stop_before_queue(self):
(self.article / 'a.webp').write_bytes(b'webp')
with self.assertRaisesRegex(ValueError, '暂不支持'):
self.store.enqueue(self.article)
self.assertEqual(self.store.state()['jobs'], [])
def test_missing_and_outside_image_rejected(self):
for image in ('missing.png', '../outside.png', 'https://example.com/a.png'):
self.md.write_text('# 标题\n\n正文 ![](' + image + ')\n\n来源:测试,如有侵权,请联系删除。')
with self.assertRaises(ValueError):
self.store.enqueue(self.article)
def test_no_duplicate_queue_or_cross_tab_claim(self):
job = self.begin()
self.assertIn('已在队列', self.store.enqueue(self.article))
self.assertEqual(self.store.next({**self.owner, 'owner': 'tab-b'})['action'], 'other_tab')
self.assertEqual(self.store.next(self.owner)['job']['id'], job['id'])
def test_stop_invalidates_old_continuation(self):
job = self.begin()
epoch = self.store.run['epoch']
self.store.control('stop')
self.store.control('start')
with self.assertRaisesRegex(ValueError, '过期'):
self.store.event(dict(owner='tab-a', id=job['id'], state='importing', epoch=epoch))
def test_restart_pauses_and_preserves_claim(self):
job = self.begin()
restarted = Store(self.root / 'queue.sqlite3')
self.assertFalse(restarted.run['running'])
self.assertEqual(restarted.run['active'], job['id'])
self.assertEqual(restarted.run['batch']['target'], 1)
def test_accept_requires_manual_submission(self):
job = self.begin()
with self.assertRaisesRegex(ValueError, '人工提交'):
self.accept(job)
def test_accept_is_atomic_and_idempotent(self):
job = self.begin()
self.submit(job)
self.accept(job)
self.accept(job)
self.assertEqual(self.store.run['batch']['count'], 1)
self.assertIn('已受理', self.store.enqueue(self.article))
with self.assertRaisesRegex(ValueError, '已受理'):
self.store.resolve(dict(id=job['id'], confirmed_not_accepted=True, retry=True))
def test_unknown_never_requeues_automatically(self):
job = self.begin()
self.submit(job)
self.transition(job, 'submit_unknown')
self.assertEqual(self.store.next(self.owner)['job']['state'], 'submit_unknown')
self.assertEqual(self.store.run['batch']['count'], 0)
self.assertIsNone(self.store.run['switch'])
def test_migrate_real_ledger_shape(self):
record = dict(markdown_sha256=hashlib.sha256(self.md.read_bytes()).hexdigest(), block_repost=True,
markdown_path=str(self.md), status='accepted_pending_moderation', post_id='12345')
self.store.import_history([record])
self.assertIn('已受理', self.store.enqueue(self.article))
def test_switch_only_after_target_and_with_more_work(self):
job = self.begin()
self.submit(job)
self.accept(job)
other = self.root / 'other';shutil.copytree(self.article, other)
(other / 'a.md').write_text(self.md.read_text().replace('正文内容', '另一篇内容'))
self.store.enqueue(other)
self.assertEqual(self.store.next(self.owner)['action'], 'switch')
self.assertEqual(self.store.next(self.owner)['action'], 'switch')
self.store.switch_begin({'owner': 'tab-a'})
with self.assertRaisesRegex(ValueError, '禁止重复'):
self.store.switch_begin({'owner': 'tab-a'})
self.assertEqual(self.store.next(self.owner)['action'], 'switch_wait')
next_job = self.store.next({**self.owner, 'account': {'uid': '456', 'name': '新测试账号'}})
self.assertEqual(next_job['action'], 'prepare')
self.assertEqual(self.store.run['batch']['count'], 0)
def test_no_switch_when_queue_done(self):
job = self.begin();self.submit(job);self.accept(job)
self.assertEqual(self.store.next(self.owner)['action'], 'idle')
self.assertIsNone(self.store.run['switch'])
def test_bad_result_does_not_increment(self):
job = self.begin();self.submit(job)
with self.assertRaises(ValueError):
self.store.accept(dict(owner='tab-a', id=job['id'], post_url='https://example.com/thread/12345.html'))
self.assertEqual(self.store.run['batch']['count'], 0)
if __name__ == '__main__':
unittest.main()
+22
View File
@@ -0,0 +1,22 @@
# 本轮验证记录
日期:2026-10-31。测试均未使用真实账号,未向新闻 API 请求内容,未向论坛上传或发布。
| 范围 | 结果 |
| --- | --- |
| Python 下载器、队列与 HTTP 接口 | 27/27 通过 |
| JavaScript 导入器与停止/覆盖保护 | 11/11 通过 |
| 连接桥旧响应与终止竞态 | 1/1 通过 |
| 本地前端 | 配置保存、开始/停止、桌面与手机布局通过,无页面脚本错误、无手机横向溢出 |
| 模拟编辑器 | 导入→排版→待人工审核;来源唯一、重复排版保持一致;程序无自动发帖点击 |
| 模拟人工发表与未知结果 | 真实测试点击被记录;候选结果页未适配时暂停,不调用受理接口 |
| 模拟换号插件 | 启动请求尚未返回时终止,返回后调用暂停,不执行登录;状态明确 paused |
| JavaScript 语法 | 合并脚本、连接脚本、更新后的插件 content.js 通过 |
浏览器使用本机 Chrome 的独立临时配置。对论坛域名的所有请求均被测试 HTML 拦截。后端使用临时目录和 SQLite,不写入正式任务库。
页面截图:`artifacts/desktop.png`、`artifacts/mobile.png`。
测试中曾发现模拟网页未声明 UTF-8,导致中文登录入口无法匹配;已修正测试响应编码并重新通过。真实网页编码未由此假定。
尚未验证:用户浏览器中油猴本地通信权限、真实网站当前的导入与附件转换、真实换号账号链路、结果页受理/审核/公开状态。按用户决定,结果页 HTML 待提供,自动核对暂不实现;因此当前不宣称全流程批量处理已完成在线验收。
@@ -0,0 +1,167 @@
# 广谈大湾网:下载、导入排版、人工发布与换号
本文中的帐号名称和用户编号已匿名化;真实记录保存在本地排除目录中。
记录日期:2026-10-31(北京时间)。本文依据用户指导、Chrome 实际操作和现有源码检查编写。它是后续实现的流程依据;需求不等于功能已经实现。
## 1. 不可改变的边界
- **“发表帖子”只能由用户亲手点击。** 自动化只下载、选择、上传、填写、排版、校验、提示审核、观察提交结果和记录计数。禁止模拟点击发布、调用提交函数、直接发送发帖请求或通过快捷键提交。
- 标题被截断、正文或来源被修改后,都必须重新进入人工审核。发生标题过长错误后,“重新尝试发表”也由用户再次点击。
- 用户决定分区和文章根目录;“理财财经”只是本次例子,不能成为所有文章的固定分区。
- 全流程页面常驻“终止流程”按钮。点击后立即取消后续动作、重试、队列领取和自动换号,不自动恢复。已经到达网站的请求不能撤回,需保留记录并核对。
- 已发表或已被网站接收待审核的文章不能再次排入发布队列,换账号也不能绕过去重。
## 2. 本次演示记录
| 项目 | 结果 |
| --- | --- |
| 站点 | https://www.dww11.com/ |
| 当前账号 | 示例用户,页面用户编号 <已匿名> |
| 分区 | 理财财经,fid=125 |
| 入口 | 首页右上角“免费发布信息”→ 分区页“理财财经” |
| 分区页 | https://www.dww11.com/special/post/ |
| 新帖页 | https://www.dww11.com/forum.php?mod=post&action=newthread&fid=125 |
| 标签行为 | 点击分区后新开标签,原分区页保留 |
| 测试标题 | 刚靠新电影赚23亿票房,周星驰就卖掉内地影院 |
| 原标题长度 | 22 个 Unicode 码点;本篇无需截断。网站计数方式尚未实测 |
| 目录 | `<项目根目录>/tanshuapi_news_downloader/downloads/刚靠新电影赚23亿票房,周星驰就卖掉内地影院` |
| 目录内容 | 1 个 Markdown + 26 张 JPG,共 27 个文件 |
| 来源 | 洲洲影视娱评(取文章末尾来源,不取正文“信息来源”截图) |
| 导入 | 已完成;面板提示“已填写标题、正文和 26 张配图” |
| 排版 | 已完成,正文 4 号黑色,尾注 1 号银色 |
| 核对 | 编辑器 DOM 中图片 26 张,最终来源尾注仅 1 条 |
| 发布 | 用户已亲手点击;网站已受理,详情页显示“审核中” |
| 帖子链接 | https://www.dww11.com/thread/23697411.html |
| 帖子标识 | 主题 ID 23697411;首帖 PID 238766 |
| 数量 | 已明确受理 1 篇;已确认审核通过并公开 0 篇 |
| 发布后图片 | 正文 21 张 + 附件区 5 张,共 26 张 |
| 换号 | 未执行;本次未设置最小值和最大值 |
本次页面没有“终止流程”按钮。用户明确同意本次演示暂用浏览器控制条停止,常驻页面按钮仍是后续自动化的必备条件。该例外只适用于本次演示。
本次已验证导入、排版、用户亲手发表以及网站受理后的帖子详情页;页面明确显示“审核中”。尚未验证审核通过后的访客可见性、标题超限错误或换号链路。用户称发表后自动回首页,但本次实际读取的原编辑器标签已导航至帖子详情页,不能将“提交后必然自动回首页”写成网站行为。应在结果登记后主动回首页并核对。
## 3. 用户操作步骤与自动化对应
1. 启动前让用户选择分区、下载目录、待处理文章范围,并填写每个账号发布数量的最小值和最大值。展示当前账号。先初始化常驻终止按钮与持久任务记录。
2. 从下载器接收完整落盘的文章目录。检查正文、图片引用、来源和去重记录;保存中或缺文件的目录不进入导入队列。
3. 点击“免费发布信息”,展示实际可用分区,由用户选择。等待新标签打开并核对分区名称及 fid;不能继续操作旧标签或仅凭坐标判断。
4. 在右下角“本地文章导入 · 1.0”选择“选择文章目录”。日常演示选择单篇目录,目录应包含 Markdown 和全部配图;不要只选 `.md`。如果选择根目录导致出现多个 Markdown,必须明确选中哪一篇。
5. 在系统文件选择器进入用户指定根目录,再选择对应文章文件夹。系统“Upload”是把所选文件提供给页面读取,不是发表帖子;浏览器另有目录文件数量确认。
6. 阅读导入面板标题、正文预览、图片数量和告警。点击“上传配图并填写”后才开始向网站上传正文引用的图片。等待明确完成提示,不能用固定等待时间代替完成判断。
7. 读取文章最后的标准尾注 `来源:名称,如有侵权,请联系删除。`,只取名称。本篇是“洲洲影视娱评”;用户先前茅台文章例子是“酒业内参”,不能硬编码。
8. 点击编辑工具栏绿色“一键排版”(不是网站自带“自动排版”,也不是“一键导入微信公众号并排版”)。实际弹窗要求输入**来源公众号名称**,并非标题。本篇填“洲洲影视娱评”,确认后会提示排版完成。
9. 标题由本地导入器填入页面标题框;仍需核对完整标题。当前“一键排版”不弹标题输入框。如果用户说“输入标题”,应在标题框填写和确认,而不是填到来源弹窗。
10. 校验标题、正文、图片数量和顺序、尾注唯一性、账号和分区。保留用户原文;本篇的发布时间、“打开网易新闻 查看精彩图片”等原始文字未擅自清理。需要清理时应成为用户可见的规则。
11. 页面提示“已完成,等待人工审核,请自行点击发表帖子”。自动化停止修改编辑器,保留终止按钮,等待用户操作;不自动保存草稿或发表。网站原有自动保存仍可能运行。
12. 用户自行点击“发表帖子”。自动化只观察结果。若标题超长,按第 5 节修正,重新提示审核,等待用户再次点击。
13. 收集帖子链接/编号、作者、分区和结果证据后记账。审核中单独记录并阻止重发;结果不明时先核对,不能自动重试发布。
14. 已明确成功后先持久保存发布记录与计数,再回到首页。达到当前账号目标篇数且没有未决提交时,点击页面顶部“登陆下一帐号”,核对登录成功后再处理下一篇。
## 4. 页面适配依据与当前缺口
以下选择器来自本次可见页面或本地源码,未来运行时须重新检查,不能保证网站改版后仍成立。
| 控件 | 当前依据 | 注意事项 |
| --- | --- | --- |
| 当前用户 | `#mumucms_username` | 同时读取页面用户名与用户链接编号 |
| 下一账号 | `#dww11-extension-next` | 完成记录后才允许换号;核对成功,不以按钮点击作为成功 |
| 标题 | `#subject` | 本次读取到 maxLength=-1,不能只依赖 HTML maxlength |
| 可视化正文 | `iframe#e_iframe` 内的 body | 图文与尾注检查应在 iframe 内进行 |
| 纯文本开关 | `#e_switchercheck` | 不将 Markdown 直接当网站 BBCode |
| 一键排版 | `#dww11-format-oneclick` | 原生 prompt 输入来源,随后 alert 提示完成 |
| 发表 | `#postsubmit` | 仅用于识别状态和观察人工提交,禁止自动触发 |
| 导入面板 | 油猴脚本面板,包含 choose/import 等控件 | 面板可能位于 Shadow DOM;普通页面选择器未必能定位,优先按可见控件操作 |
本次实际遇到的兼容问题:
- 浏览器连接器直接设置目录文件被“允许访问文件网址”权限阻止。本次没有更改扩展权限,改用 Chrome 原生系统选择器完成。后续实现应优先复用本地文章包接口,减少对文件对话框的依赖。
- macOS 路径含中文,逐键输入曾丢失中文;通过文本框完整赋值成功定位。输入后必须核对完整路径,不能只看末级数字或英文。
- 导入器能完成 26 张图片上传,并临时禁用发表按钮;上传完成后恢复。上传过程中不允许换号。
- **来源重复:** 一键排版保留了导入正文中的原尾注,又追加银色尾注。本次在编辑器中删除原来的那一条,保留银色尾注,复查最终只出现一次。后续应在排版适配器中精确处理末尾标准尾注,不能全局删除正文中的“来源”。
- 页面“还可输入 80 个字符”在导入后仍显示 80,不能将这个显示值当作可靠的标题长度校验结果。
- 页面有“上次未提交成功的数据”恢复提示;本次没有点击恢复,避免把旧稿混入新稿。后续遇到已有正文或恢复冲突必须暂停处理。
## 5. 标题超过 80 字符
保留 `original_title`,派生 `submitted_title`。超过站点限制时截取前缀并追加字面量 `...`,总长不超过 80;即长度允许时保留前 77 个字符再加 3 个点。示例中的“标题文本...”表示实际标题前缀和省略号,不额外加入中文括号。
当前站点的“字符”计数方式尚未通过超长标题实测。实现时兼容中文和 emoji,不能切断代理对;先按保守规则校验,再根据站点明确错误调整。若站点按字节计数,则按其实际字节预算截取,不能反复盲目减少。
现有导入器在读取阶段对超过 80 个 Unicode 码点的标题直接报错。后续需把截断放在导入前,否则到不了人工审核阶段。截断不覆盖原 Markdown 标题,也不改变文章去重身份。
如果用户点击发表后才收到标题超限提示:确认网站明确未接受 → 修正标题 → 显示新旧标题 → 重新人工审核 → 用户再次点击。任何自动修正都不等于用户已批准发表。
## 6. 发布记录、去重与恢复
建议独立 SQLite 发布任务库,与下载去重库分开;下载成功不能视为发布成功。本次先保存演示任务 JSON,尚未实现自动数据库或网站历史帖子同步。
稳定文章身份优先采用下载器原始文章 ID 或规范化源 URL,同时保存正文与图片内容指纹用于发现重新下载/改名副本。禁止只按本地目录名、截断标题或当前账号去重。默认整个站点跨账号、跨版块不重复发表同一篇;如以后需要重发,必须由用户显式决定。
建议字段:article_key、source_url/source_id、content_hash、原始/最终标题、目录、来源、状态、账号 UID、fid、附件映射、排版版本、人工提交时间、提交尝试编号、帖子 ID/URL、网站受理/审核/公开状态、错误、创建和更新时间。日志不保存密码、Cookie 或上传凭证。
主状态:
```text
downloaded → queued → importing → formatting → awaiting_user_review
→ user_submitting → published / accepted_pending_moderation
```
分支:`stopped`、`needs_attention`、`submit_unknown`、`rejected`。未决提交不得重新上传或领取同一文章。单批次串行,跨标签领取加持久锁;刷新后先恢复状态,不自动启动。
去重记录需要覆盖用户已发表帖子:优先导入原有记录,或从“修改/删除信息”进入本人帖子列表进行核对。仅标题相同只能标记疑似重复,不能据此自动认定是同一正文。本次未扫描历史帖子,后续批量运行前必须完成历史对账;本次文章已记为“网站已受理、审核中”,已有帖子 ID 与 URL,必须阻止重发;不能误写为等待用户点击,也不能写为审核通过。
成功记录与计数使用同一个事务:帖子唯一键(站点+帖子 ID)防止页面刷新重复加一。先记录再返回首页。遇到网络超时、页面关闭或不明确提示,记录 `submit_unknown`,人工/页面核对后再决定,禁止自动重复发表。
网站明确受理待审核时,记 `accepted_pending_moderation` 并锁住文章。本规范建议把明确受理计入账号本轮提交数量一次,另存公开发表数量;后续审核通过只改变公开状态,不再加本轮数量。本次已观察到网站审核中状态,因此本地记录明确受理数为 1、已确认公开数为 0;批量自动记账尚未实现。驳回不自动补发或重投。
## 7. 每个账号随机篇数与换号
用户填写整数 `min`、`max`,要求 `1 ≤ min ≤ max`。首次为当前账号创建发布批次及每次确认换号成功后,从闭区间 `[min,max]` 均匀抽取整数 n,保存目标和当前计数。min=max 时固定该数量。不要按每篇重新抽 n,刷新、暂停或重启也不重抽。
- 本轮计数只含当前发布批次明确成功/明确受理的唯一文章,不含上传、排版、等待人工审核、失败和结果不明。
- 达到 n 后先保存记录与 `switch_pending`,回首页,核对没有待提交草稿,再点击“登陆下一帐号”。
- 观察并确认实际 UID/用户名变化且登录成功,再建立下一账号批次、重置批次计数并抽新 n。不可在不确定结果时重复点击换号。
- 验证码、账号异常、频率限制、登录失败或账号池无可用账号时暂停,保留当前文章与计数;不把受限文章换账号重投。
- 用户手动改变账号时立即暂停,重新关联批次与上传附件;不能沿用旧账号附件上下文。
- 队列已经处理完时直接结束,不为“凑满 n”重复文章,也不无意义换号。
本次没有给定 min/max,所以不随机设置目标,也不执行换号。账号扩展现有自己的抽号/登录持久记录,应复用其登录流程,不另存账号密码。
## 8. 常驻终止按钮的实现要求
按钮在首页、分区页、新帖页、结果页、换号页面保持可见,不依赖折叠的导入面板。跨标签共享同一批次停止状态,导航后先恢复按钮和停止状态再执行任务。
点击立即写入持久 `stopped` 并提升运行版本号,取消定时器、队列和可取消的上传/下载请求;每个异步步骤返回时检查停止标志和版本,过期回调不得再填写、跳转或换号。浏览器自带模态弹窗会阻挡网页按钮,因此自动流程应把来源/审核提示改成页面内面板,保留停止入口。
已被服务器接收的图片/帖子不能被停止按钮撤回。停止后允许被动记录已在途请求的最终结果,但不得继续业务动作;需要清楚显示“已停止,已发出的请求可能仍已完成”。人工终止后必须由用户明确重新启动,不能因刷新或服务重启自动恢复。
## 9. 后续实现与验收清单
1. 优先实现常驻终止按钮、持久队列、站点级去重和历史发布对账,再接批量导入。
2. 改造导入前标题截断、唯一来源尾注和可重复执行排版;保留原文与本地文件。
3. 建立本地任务服务与浏览器执行器之间的配对通信,只传当前文章所需数据,不传下载 API Key 或账号库。
4. 人工发布边界验收:没有用户点击时,等待任意时间都不会发表;标题修正后仍等待新的用户点击。
5. 验证缺图、上传部分失败、重复来源、长标题、emoji、多 Markdown、非空编辑器、人工修改、刷新/断网、多标签竞争。
6. 用户实际发布后,补录成功页或审核中页面证据、帖子 URL/ID,并验证重复打开不会增加计数。
7. 给定 min/max 后验证 n 范围、每账号仅抽一次、达到目标换号、核对新 UID、恢复不丢计数。
8. 在下载、上传、排版、等待审核、观察结果和换号各阶段点击终止,确认不再触发后续动作。
## 10. 当前交接
用户已亲手点击发表。原编辑器标签现为帖子详情页,标题与作者“示例用户”一致,面包屑为理财财经,显示“审核中”。已保存主题 ID、首帖 PID、URL、受理数量与去重记录,禁止再次发同一文章。登记后已点击网站首页入口,并核对 URL 为 `https://www.dww11.com/`,当前账号仍为示例用户(UID <已匿名>);没有设置 min/max,因此未换号。
发布结果检查发现正文内有 21 张图片,另外 5 张在附件区:image009.jpg、image014.jpg、image016.jpg、image025.jpg、image026.jpg,总数 26 张;来源尾注仅一条。发布后的正文与初次待审稿存在差异(例如原发布时间及“打开网易新闻 查看精彩图片”文字已不在结果正文中)。本次未记录用户审核期间的具体编辑动作,无法将差异归因于用户修改、编辑器同步或网站转换;后续需在用户点击的时刻保存最终待提交快照,再与结果对照。不得自动编辑或重发这篇已受理的帖子。
截图:`test/history/周星驰文章_待人工审核.jpg`(历史待审稿)、`test/history/周星驰文章_网站审核中.jpg`(网站受理结果)。演示任务:`test/history/演示任务记录.json`;站点级去重登记:`发布去重记录.json`。记录保存本次结果,不代表完整历史同步或批量程序已经实现。
## 11. 发布结果判断与本次补录
- 用户点击后,不按“回到首页”判断成功;先记录实际跳转 URL 和页面结果。
- 本次 URL 为 `/thread/23697411.html`,页面标题、理财财经面包屑、楼主示例用户及“审核中”共同确认网站已受理。DOM 首帖容器为 `#postmessage_238766`。
- 网站审核中并非失败,不能为追求公开状态而再次提交;也不能仅凭当前登录用户能看到详情,宣称所有访客可见。
- 刷新、重复读取或后来审核通过时,都用站点+主题 ID 保证受理数量只计一次。
- 本次没有出现标题超限,相关处理仍是待验证需求。
- 完整历史对账、常驻终止按钮、随机数量和自动换号仍属后续实现及验收范围。
+178
View File
@@ -0,0 +1,178 @@
# 新闻下载、导入排版、人工发表与换号实施计划
本文中的帐号名称和用户编号已匿名化;真实记录保存在本地排除目录中。帖子编号仍用于说明既有受理证据,不代表当前公开状态。
更新日期:2026-10-31(北京时间)。本版整体替换早期自动发帖设想,依据当前源码、[发布流程操作记录与自动化规范](发布流程操作记录与自动化规范.md)、[演示任务记录](演示任务记录.json)和[发布去重记录](发布去重记录.json)重新梳理。
本次进行了本地源码与记录核对,未重新操作线上页面、下载新闻、上传图片或发表帖子。下文“演示已验证”引用既有操作记录;“源码已有”不等于最新版本已在线上重新验收。
## 1. 当前目标与确定规则
目标是:自动下载、选文、上传、填写、排版和校验,等待用户亲手发表,然后观察结果、记录数量,达到账号目标后换号,继续下一篇。
- **发表及重试发表必须由用户亲手点击。** 不模拟点击、不调用提交函数、不直接发送发帖请求、不通过快捷键提交。自动化没有发布接口。
- 用户选择分区、文章根目录、处理范围及每账号篇数 min/max;理财财经 fid=125 是演示样例,不硬编码为全部任务的目标。
- 标题、正文或来源修改后重新进入人工审核;等待审核期间自动化不继续修改编辑器。
- 全流程常驻“终止流程”按钮,跨标签、导航和换号保持生效;停止后不因刷新或服务重启自动恢复。
- 同一文章在整个站点跨账号、跨版块去重。已公开发表或已受理待审核均阻止再次入队;结果不明时先核对。
- 新下载文章不额外插入接口发布时间和分隔线。历史文件及源正文中原有的时间、文字不因此自动删除。
结论:工作流可行,单篇人工链路已有实机记录,但持久队列、本地桥接、统一终止、自动计数和与换号扩展联动仍需实现。实施重点从“寻找全部页面元素”转为“连接已有组件并保证停止、审核、去重和结果核对”。
## 2. 本轮变化与现状
| 模块 | 当前确认 | 后续工作 |
| --- | --- | --- |
| 排版脚本 `dww11-wechat-oneclick.user.js` v1.0.9 | 新增独立“一键排版”;`existingSource()` 识别文末来源/作者及版权行;`runFormat()` 取出原尾注后交给 `formatBody()` 重新排版;无来源才弹窗补充 | 提供可等待、可取消、无模态弹窗的程序接口;验证复杂尾注与编辑器转换后重复执行 |
| 下载保存 `article_writer.py` | 标题后直接接正文;不再插入接口 time;移除无文字且无图片的空标题节点;末尾仍生成标准来源尾注 | 输出持久文章清单与结构化来源/源网址/指纹,建立完整保存后的入队衔接 |
| 下载调度 `app.py` | 频道、分页、下载去重、进度、停止;历史库仍只有指纹到目录映射;状态接口含汇总、日志、输出目录 | 独立发布队列、每篇完成登记、重启对账;不能拿日志当可靠文章队列 |
| 导入器 v1.0.0 | `stagePackage()`、`importPrepared()`、附件上传及登记、编辑器填写校验 | 接入取消信号、无弹窗冲突状态、持久恢复;导入前处理长标题 |
| `dww11_make_package.py` | 已能生成 v1 Markdown/base64 图片包 | 包大小限制、当前任务按需传输、引用完整性校验 |
| 换号扩展 v3.0.4 | 自有账号库、抽号、登录、暂停与持久流程;下一账号按钮已有 ID | 接收统一终止、输出可核对的换号结果,与发文批次计数协调;不能假定网站脚本能直接调用扩展内部路由 |
| `auto_news_upload/` | 操作规范和两份 JSON 记录已存在 | 尚无完整自动执行程序或 SQLite 发布任务库 |
新版排版的“保留来源”指保留识别到的尾注文字;内部会规范化空白并重建为纯文本格式节点,不保证原 HTML、链接或空格逐字不变。自动化应展示提取结果,疑似误识别时暂停。
旧操作记录中“每次输入来源”和“排版后手工删除重复来源”属于当时演示行为。当前 v1.0.9 的独立排版入口已针对已有来源提供处理,不能再把这一功能整体列为未实现;仍需回归验证。微信 URL 采集入口仍走另一条流程,本项目只接独立排版逻辑。
下载器测试文件已含不插入发布时间的断言,本次未重跑测试。新下载文件的时间格式已明确,不再列为待确认项;这不授权清理历史稿件。
## 3. 已有实机证据与尚未解决的问题
既有演示记录:账号示例用户(UID <已匿名>),理财财经 fid=125,文章“刚靠新电影赚23亿票房,周星驰就卖掉内地影院”,来源“洲洲影视娱评”。用户亲手发表后,网站返回主题 23697411、首帖 PID 238766,记录时显示审核中。
- 已明确受理 1 篇;已确认公开 0 篇。不得根据旧记录推断现在已审核通过,也不得重发这篇文章。
- 导入阶段编辑器有 26 张图片;结果页正文 21 张、附件区 5 张。总数齐全不能证明正文位置和顺序正确。
- 附件区图片为 image009.jpg、image014.jpg、image016.jpg、image025.jpg、image026.jpg。需要以附件 ID 与正文引用逐一对照,不只比较文件名或总数量。
- 结果正文与待审稿有文字差异;记录未覆盖用户审核期间编辑动作,不能归因于网站转换或用户操作。
- 本次未重新读取线上结果页;原演示未验证审核后的访客可见性、超长标题报错、随机数量与换号链路。
后续准备完成时保存基线快照,用户亲手点击时再捕获最终标题、编辑器正文、图片引用和附件映射,记录内容修订号与提交尝试。持久化可能来不及完成时要能识别缺失证据;不能为补记状态而代替用户重新提交。发布后分别检查受理状态与内容完整性,内容异常只标记待处理,不自动编辑或重发。
## 4. 目标工作流与架构
```text
用户设置分区、根目录、处理范围、min/max
→ 初始化终止按钮与持久批次
→ 历史去重对账
→ 已完整下载文章校验 / 持续接收新文章
→ 单篇领取与文章包准备
→ 确认新帖标签、账号、分区
→ stagePackage → importPrepared
→ 独立一键排版 → 标题/正文/图片/尾注检查
→ 等待人工审核与用户亲手点击发表
→ 只观察提交结果
→ 原子登记受理记录、去重与本轮计数
→ 回首页并核对
→ 未达目标:下一篇;达到目标且仍有任务:换号并核对
→ 队列结束:结束批次
```
建议继续采用“本地 Python 任务服务 + 浏览器执行器 + 网站适配层”,复用导入器与换号扩展。第一步验证油猴特权请求与回环服务通信;若目标浏览器不兼容,再改为扩展桥接,不同时维护多套主链路。
本地服务仅监听回环地址,使用配对凭据和限定接口;只向网站传当前文章内容,不传下载 API Key 或账号库。不能把现有下载器 `/api/session` 直接当作对任意网页开放的任务授权入口。导入器是 `@grant none`,排版器有特权权限,必须实测页面与脚本隔离上下文间的调用,不能简单假定合并后仍可访问原来的 `window` 对象。
文件传递优先复用文章包,不依赖自动操作系统目录选择器。原生选择器可作为人工导入备选。同一版块的点击可能打开新标签,任务必须绑定实际标签及导航版本,不能在旧分区页继续执行。
电脑、本地服务及浏览器需保持运行;没有新增定时无人值守发表需求。发布始终需要用户点击。新任务的下载 Key 仍按现有方式在内存中使用,不默认落盘。
## 5. 数据接入、格式与来源
### 完整落盘与元数据
复用下载器现有保存及去重逻辑,新增每篇成功结果清单,至少保存稳定身份候选、原始标题、来源、源 URL、目录、Markdown 路径、图片清单和校验值。新发布记录写入失败时可通过清单对账补录,不应为回滚发布登记而删除完整文章。
当前保存器先创建最终目录,再逐个移动文件,扫描到目录并不代表下载结束。新下载采用成功回调/完成标记与清单协调入队;旧目录在确认非保存中且引用完整后显式导入。下载历史库失败时原下载器自有清理行为,与未来发布登记失败处理分开。
### 标题
保留 `original_title`,派生 `submitted_title`。超过 80 字符时按允许的计数预算截取前缀并追加字面量 `...`,不能截断 emoji 代理对;站点实际按码点、UTF-16 或字节计数仍需实测。当前页面曾读到 maxLength=-1,“还可输入80个字符”也未随填入更新,不能作为可靠校验。
截断必须发生在导入器 `compile()` 之前。现有 `preparePackage()` 已接受 `data.title`,可以由适配层提供派生标题,不改原 Markdown;现有打包器本身尚未生成这个覆盖值。显示新旧标题并进入人工审核。若人工提交后收到明确超限拒绝,修正后仍由用户再次点击;结果不明时不改稿重投。
### 来源与正文
复用 v1.0.9 的 `existingSource()` 与 `formatBody()`,保留 4 号黑色正文、1 号银色尾注、图文间隔及图片适配。识别末尾来源时保留整段来源/版权文字,不只抽出名称再拼回默认句子。没有来源则在页面内要求补充,不能让原生 prompt 阻挡终止按钮。
需要验证:标准下载尾注、多行版权说明、重复尾注、正文中出现“来源”、来源后有图片、嵌套 FONT、纯文本换行、编辑器来回切换、重复排版。不得全局删除正文中的来源文字;原稿本身已有多条来源时应提示核对,而非宣称新函数能语义去重。
正文清理只沿用已确认规则;不自动删除历史发布时间、“打开网易新闻 查看精彩图片”等文字。下载器新生成格式与旧文件格式并存,文章身份不能仅靠整个 Markdown 的精确哈希判断。
### 图片与编辑器
下载器支持 WebP/AVIF/SVG,打包器和导入器仍只支持 JPG/JPEG/PNG/GIF。先在入队前拦截不兼容格式,不能等打包时静默漏掉;如后续增加转码,生成派生副本并重写引用,保留原文件。上传大小同时遵守导入器 20 MB 和网站当前限制。
校验顺序为:本地引用完整 → 上传附件返回编号 → 网站附件登记 → BBCode/可视化图片关联 → 排版前后附件引用保持 → 人工提交时最终快照 → 结果页正文与附件区对照。现有缓存仅在页面内有效,持久复用必须绑定账号与上传上下文并重新验证。
遇到旧稿恢复提示、非空编辑器或上传期间人工修改,暂停并保留现场。导入器现有 confirm 弹窗需要转换为页面内状态;不能自动确认覆盖。
## 6. 持久状态、发布去重与观察
独立 SQLite 任务库与下载历史库分开。启动前先迁入两份现有 JSON 的已受理记录,保证主题 23697411 不再入队;保留原 JSON 作为证据。再核对原有发布记录或本人历史帖子列表。标题相同只标记疑似重复,不能据此确认同一正文。完整历史对账尚未完成,不能把这一个样本当全量历史。
稳定身份优先关联原始文章 ID/规范化源 URL,同时保存正文与图片指纹、文件版本哈希和身份别名。当前下载历史只有指纹与目录,原始 URL/来源并未形成完整持久清单;历史稿无法恢复的信息要标为未知,不能猜造。跨账号、跨版块去重,改名、标题截断、删除后重新下载均不改变已受理身份。
建议记录:article_key、身份别名、source_url/source_id、内容及图片指纹、原始/提交标题、来源尾注、目录、账号 UID、fid、附件映射、排版版本、内容修订号、任务状态、批次 ID、领取租约/执行版本、尝试 ID、用户点击时间、最终快照、帖子 ID/URL、受理/审核/公开状态和错误。
```text
downloaded → queued → importing → formatting → awaiting_user_review
→ user_submitting → published / accepted_pending_moderation
```
异常分支为 `needs_attention`、`submit_unknown`、`rejected`、`stopped`。其中 `published` 仅在有公开发表证据时使用;提交意图或点击事件不是网站受理证据。人工修改后记录新修订,继续等待人工审核。
- 自动化只监听用户操作及网站结果,不主动触发提交。点击前预登记候选尝试,点击时关联最终快照;即使快照写入失败,也不能自动补交。
- 结果需综合帖子 ID/URL、作者、分区、标题和受理/审核提示;跳转到首页、HTTP 200 或按钮变灰均不能单独证明成功。
- 以“站点+帖子 ID”唯一键登记受理记录,并在同一事务更新去重与批次计数。重复打开、刷新及后续审核通过不重复加一。
- 审核中计入明确受理数一次,公开数量另记;驳回不自动补发。结果页内容异常单独记录,不撤销受理去重。
- 网络中断、关闭标签或证据不足进入 `submit_unknown`,先对账;租约过期也不能重投未决文章。
- 多标签单篇互斥;刷新或服务重启先显示恢复状态,等待用户明确继续。已停止批次不得自动恢复。
## 7. 账号目标与统一终止
用户填写整数 `1 ≤ min ≤ max`。首次建立当前账号批次、以及确认换号成功后,从闭区间均匀抽取目标 n 并持久化。每账号批次只抽一次;暂停、刷新和重启不重抽。没有 min/max 时不能擅自设值或启动自动换号。
本轮只计明确受理的唯一文章,导入、排版、等待审核、失败和结果不明不计。达到 n 后,先完成记账,确认没有未决提交,回首页并核对,再触发现有“登陆下一帐号”。核对扩展流程结束和实际账号 UID/用户名;无法确认时暂停,不连续点换号。账号池只有一个可用账号等导致 UID 未变化时不能假装换号成功,应显示原因等待处理。
队列空时结束,不为凑数重发,也不无意义换号。账号手动变化、验证码、限制或登录异常时暂停,不带着旧附件上下文继续。账号密码继续由现有扩展管理。
统一终止必须覆盖首页、分区页、新帖页、结果页和换号页面。点击先持久写入 stopped 并更新执行版本;立即禁止新领取、下一张上传、排版写入、跳转和换号,取消可取消请求。所有异步返回重新检查停止状态,过期回调不得继续。
**现有组件的停止并不贯通:** 下载器停止是等待当前请求/保存结束;导入器没有对外取消接口;换号扩展有独立流程;排版仍使用模态弹窗。仅在外层增加按钮不能满足全流程终止。需要给这些模块增加取消/暂停衔接,尤其要在扩展开始退出/登录前以及后续各步骤检查共享停止状态。不能把扩展内部消息接口当作已公开的油猴 API。
停止不撤回已经发出的请求,允许被动核对其最终结果,但不继续业务动作。原演示使用浏览器控制条停止的例外不延续至正式实现。
## 8. 页面适配依据与剩余信息
| 环节 | 已有依据 | 仍需核对 |
| --- | --- | --- |
| 分区入口 | `/special/post/`;演示选择后新开标签 | 用户实际目标分区及各区必填项,不固定 fid=125 |
| 当前账号 | `#mumucms_username`,用户名/用户链接编号 | 登录失效与换号成功的稳定判定 |
| 标题/正文 | `#subject`、`#e_textarea`、`#e_iframe`、`#e_switchercheck` | 当前网站编辑器同步与标题计数 |
| 导入 | `window.DWW11Importer`,面板在 Shadow DOM | 实际浏览器中程序接口与停止信号 |
| 排版 | `#dww11-format-oneclick` | v1.0.9 来源识别与发布后往返结果;目前没有公开排版 API |
| 发表 | `#postsubmit`、`#postform` | 只观察人工操作;失败、超限、审核及成功提示适配 |
| 受理结果 | `/thread/23697411.html`;样本首帖 `#postmessage_238766` | 从当前结果提取动态帖子 ID/PID,不硬编码样本 |
| 换号 | `#dww11-extension-next` | 扩展完成/暂停状态及全局终止联动 |
| 历史对账 | “修改/删除信息”进入本人帖子列表 | 列表 URL、分页、审核项与重复证据 |
无需重新提供已记录的完整单篇演示。下一批所需输入主要是:实际分区与文章范围、min/max、其他版块必填项、历史帖子入口及异常提示样本。标题计数、5 张图片落到附件区的原因、审核后的公开状态需要针对性验证。不需要密码、Cookie 或 API Key 值进入文档。
## 9. 实施顺序与验收
| 阶段 | 工作 | 完成条件 |
| --- | --- | --- |
| 1:基础状态与停止 | 独立任务库、现有 JSON 迁入、历史对账、统一终止状态和常驻按钮;确定各组件取消协议 | 已受理样本不可重入队;跨标签互斥;刷新不自动运行;停止后无后续动作 |
| 2:文章衔接 | 完整落盘清单、文章身份、包传输、图片检查、导入前标题派生、本地配对通信 | 新旧格式均可处理;半成品不入队;长标题不改原稿;缺图和不兼容格式明确报错 |
| 3:排版与人工审核 | 封装 v1.0.9 独立排版接口,替换模态提示,接入导入取消和编辑冲突检查 | 尾注文字保留、标准来源只出现一次;反复排版及编辑器转换无累积问题;附件引用与顺序不变 |
| 4:人工提交观察 | 提交快照、动态结果解析、审核/公开区分、唯一受理计数、正文与附件核对 | 不经用户点击永不发表;用户修改后重新审核;刷新不重复计数;结果不明不重投;定位图片落附件区问题 |
| 5:批次与换号 | min/max 持久目标、记账后回首页、复用登录流程、账号核对、终止联动 | 每批仅抽一次目标;达到目标且队列非空才换号;异常不连续换号;无旧账号附件串用 |
| 6:小批实机验收 | 覆盖故障恢复与多个页面场景 | 下载/上传/排版/待审/结果/换号各阶段终止均有效;断网、重启、多标签和人工改稿状态正确 |
重点回归场景:普通/多图/重复图片、空标题节点、多行尾注、来源后图片、重复排版、长标题与 emoji、非空编辑器、旧稿恢复、部分上传失败、上传超时、用户提交后响应丢失、审核中、驳回和换号中终止。先模拟验证,再做用户亲手提交的少量实机验收;不以已受理样本重发来测试。
## 10. 目录与本次交付范围
新实现仍放在 `auto_news_upload/`,后续规划为 `service/`、`browser/`、`tests/`、`data/` 和启动说明;当前这些实现尚未建立。保留原下载目录、原历史库及现有演示记录。复用源码时记录版本,明确安装时启用的脚本组合,避免重复工具争用编辑器。
本次仅重写本实施计划,未修改业务脚本和历史记录。历史操作规范用于追溯当时实机行为;当前组件能力以本轮核对的源码为准,人工发表、去重、终止和换号规则继续遵循操作规范。
+150
View File
@@ -0,0 +1,150 @@
# dww11 自动化流程检查与 Git 文件清单
检查日期:2026-10-31(北京时间)。本轮仅检查本地说明、源码和离线测试;没有访问网站、消耗下载 API 额度、登录、上传或发表,最初检查时未执行 Git 操作。用户随后已确认仓库范围并授权本地初始化和首次提交;下面保留检查发现,并补充本次整理结果。
## 一、检查结论
当前是可以人工衔接的组件集合,尚不是已实现的端到端自动流程。
```text
探数 Key → 下载 Markdown 和图片
→ 手工选择文章目录,或运行打包器生成文章包
→ 已登录的论坛新帖页面导入并上传图片
→ 点击独立“一键排版”并核对标题、正文、配图和来源
→ 用户审核并亲手点击发表
→ 核对受理/审核结果,登记去重,再决定下一篇或换号
```
发表与再次尝试发表均由用户亲手点击,这是现有流程规范要求。下载器、打包器、导入器、排版器和换号扩展之间没有完整的任务调度连接。
## 二、逐环节检查
| 环节 | 主要文件 | 当前能力 | 缺口或注意事项 |
| --- | --- | --- | --- |
| 下载 | `tanshuapi_news_downloader/app.py`、`article_writer.py`、`static/` | 频道、分页、正文图片落盘、下载历史去重、进度和停止 | 下载历史仅保存指纹与目录,没有供发布队列可靠消费的完整文章清单和完成标记;停止需要等待当前请求/保存结束 |
| 打包 | `dww11_make_package.py` | 将目录转换为 v1 Markdown/base64 图片包;可直接作为 Python 函数调用 | 不会自动把文章送进浏览器;仅收集 JPG/JPEG/PNG/GIF,没有完整的 Markdown 引用预检或整包大小上限 |
| 导入 | `dww11-article-importer.user.js` | 选目录或文章包、预览、按正文引用上传、附件登记、填写校验与失败回滚 | 标题超过 80 个 Unicode 码点直接拒绝;没有对外取消接口;非空编辑器确认仍使用模态弹窗 |
| 排版 | `dww11-wechat-oneclick.user.js` v1.0.9 | 独立“一键排版”,识别文末来源并重建尾注;无来源时要求补充 | 没有公开可等待/取消的排版接口,仍有 prompt/alert;复杂尾注、重复执行和发布后的图片位置仍需浏览器回归 |
| 发布 | `auto_news_upload/` 两份流程文档和两份记录 JSON | 已有单篇人工操作的历史记录 | 本目录没有完整发布执行程序、持久任务库、自动受理计数、全量历史对账或统一终止功能 |
| 换号 | `dww11-extension/` v3.0.4 | CSV 导入、扩展 IndexedDB 帐号库、抽号与登录、暂停恢复 | 未与文章任务的 min/max、受理计数和统一终止联动;扩展内部路由不能直接视作油猴接口 |
### 优先处理的问题
1. **图片格式衔接不一致。** 下载器可保存 WebP/AVIF/SVG,打包器和导入器只支持 JPG/JPEG/PNG/GIF。打包器会跳过不支持格式,导入时可能报缺图;应在打包/入队前给出明确错误,或另外实现保留原文件的转码。
2. **长标题衔接未实现。** 规范要求保留原题并派生截断标题,现导入器却在编译时直接拒绝超过 80 字的标题。需在导入前生成并展示提交标题,站点真实计数方式仍需实测。
3. **缺少可靠调度和统一终止。** 下载完成不等于可以发布;目录存在也不等于完整保存。现有模块停止行为不贯通,不能直接启动无人看管的批量链路。
4. **发布去重仍是手工记录。** 下载去重不能替代跨帐号、跨版块的发布去重。需要持久队列、未决结果核对、唯一受理登记及恢复逻辑。
5. **历史样本的图片位置有异常。** 原记录显示 26 张图片中正文 21 张、附件区 5 张,总数相同不能证明位置正确。应按附件 ID 对照最终待提交快照与结果页。
6. **旧路径已整理。** 原导入测试与打包说明引用了搬迁前目录;首次提交准备时改用 `tests/fixtures/article/` 的自造 Markdown 和 PNG,消除 legacy 依赖。
7. **扩展测试资料缺失。** 扩展 README 提到 `tests/core.test.mjs`、`tests/browser-integration.mjs`、`tests/browser-restart.mjs` 和 `测试说明.md`,当前项目中未找到,不能把文档中的历史测试声明当成本轮复验结果。
历史记录中的主题 23697411 已被网站受理并显示审核中,必须保留其本地去重证据。本轮没有核实现在是否已经公开。
## 三、运行时需要帐号或凭据的文件
下表区分“程序需要凭据才能运行”和“源码里保存真实凭据”。程序需要登录不代表源码不能进入 Git。
| 文件 | 所需帐号/凭据 | 使用和保存方式 | Git 建议 |
| --- | --- | --- | --- |
| `tanshuapi_news_downloader/app.py` | 探数 API Key | 调用新闻 API;Key 从页面传入,在运行内存使用 | 源码可纳入;不写入真实 Key |
| `tanshuapi_news_downloader/static/app.js`、`static/index.html` | 用户填写探数 API Key | 页面输入和内存变量,经本机会话接口传给服务;源码未配置固定 Key | 可纳入 |
| `dww11-article-importer.user.js` | 已登录的大湾网帐号及页面上传凭据 | 从当前页面读取上传参数,使用当前登录会话上传图片;无须写入密码 | 可纳入源码 |
| `dww11-wechat-oneclick.user.js` | 发布页面的登录会话 | 独立排版入口在当前编辑器操作;微信 URL 入口属于另一条采集路径 | 可纳入源码 |
| `dww11-extension/manager.js`、`manager.html`、`core.js` | 帐号 CSV | 选择、解析、确认导入帐号和密码 | 可纳入程序及虚拟模板,排除真实 CSV |
| `dww11-extension/db.js`、`service.js` | 扩展帐号库 | 明文密码保存在浏览器扩展来源的 IndexedDB;列表返回时去除密码 | 可纳入程序;排除浏览器资料及数据库备份 |
| `dww11-extension/background.js`、`content.js` | 当前抽取的帐号和密码 | 扩展消息传递,并填入目标网站登录表单 | 可纳入程序 |
| `today-jobs.user.js` | 可访问招聘管理列表的登录帐号/权限 | 使用当前页面和同源会话读取职位 | 辅助功能,按需纳入 |
| `dww-today-threads.user.js` | 论坛后台管理权限与页面 formhash | 使用当前管理会话执行统计查询 | 辅助功能,按需纳入 |
| `legacy/acc_switch_lite_python_legacy/server.py`、两个 lite 目录的 `import_excel.py` 与换号油猴脚本 | 旧版帐号文件、Excel 和网站登录 | 旧版实现;与当前扩展分开 | 不属于当前主链路;归档时排除真实数据 |
| `legacy/acc_switch_heavy_sqlite_legacy/init_db.py`、`import_excel.py` | 旧版帐号 Excel/数据库 | 可生成并导入保存密码的 `accounts.db` | 不属于当前主链路;数据库不纳入 |
`dww11_make_package.py` 仅处理本地文件,不需要帐号、API Key 或网站 Cookie。
## 四、已经存在的帐号数据文件:排除真实版本
| 文件 | 本轮确认 | 建议 |
| --- | --- | --- |
| `账号密码-去重可导入.csv` | 表头外 1152 条数据行、3 列;为真实帐号导入文件 | 本地保留,不纳入 Git |
| `legacy/acc_switch_lite/accounts.json` | 169 条帐号记录,169 个非空 password 字段 | 本地保留,不纳入 Git |
| `legacy/acc_switch_lite_python_legacy/accounts.json` | 169 条帐号记录,169 个非空 password 字段 | 本地保留,不纳入 Git |
以上仅统计结构和数量,不展示用户名、密码或密钥。没有执行帐号有效性验证,也没有核对 CSV 行与 JSON 记录之间的对应关系。
当前扩展运行帐号库在浏览器内,不在 `dww11-extension/` 源码目录。不要把浏览器配置目录、Cookie 导出文件或 IndexedDB 备份作为源码提交。`example.csv` 是示例模板,需保持为虚拟凭据。
## 五、拟纳入 Git 的文件清单
### A. 当前主链路源码与说明
```text
dww11_make_package.py
dww11-article-importer.user.js
dww11-wechat-oneclick.user.js
本地文章导入脚本使用说明.md
tanshuapi_news_downloader/app.py
tanshuapi_news_downloader/article_writer.py
tanshuapi_news_downloader/requirements.txt
tanshuapi_news_downloader/start.command
tanshuapi_news_downloader/README.md
tanshuapi_news_downloader/.gitignore
tanshuapi_news_downloader/static/index.html
tanshuapi_news_downloader/static/app.js
tanshuapi_news_downloader/static/style.css
tanshuapi_news_downloader/tests/test_downloader.py
dww11-extension/manifest.json
dww11-extension/background.js
dww11-extension/content.js
dww11-extension/core.js
dww11-extension/db.js
dww11-extension/service.js
dww11-extension/manager.html
dww11-extension/manager.js
dww11-extension/manager.css
dww11-extension/example.csv
dww11-extension/README.md
auto_news_upload/实施计划.md
auto_news_upload/发布流程操作记录与自动化规范.md
auto_news_upload/流程检查与Git文件清单.md
```
两份流程文档的帐号名称、UID 和电脑绝对路径已匿名化,原版保存在被排除的 `auto_news_upload/local-records/`。历史帖子编号与链接保留为受理证据说明。本文不含密码,但也引用了本地敏感文件名和数量,可按仓库公开范围进一步精简。
### B. 修复或检查后再决定
- `test-dww11-importer.cjs`:已改用仓库内自造测试稿与三张 PNG,一并纳入首次提交。
- `auto_news_upload/演示任务记录.json`、`发布去重记录.json`:属于真实运行状态。建议保留本地,另提供匿名结构模板;不因 Git 排除而删除原去重记录。
- `dww11-extension-3.0.4.zip`:本轮逐文件比对,与当前扩展目录的 11 个文件一致。可作为发行附件;日常优先跟踪源码,避免双份维护。
- `today-jobs.user.js`、`dww-today-threads.user.js`:辅助工具,不是四步主流程的必需文件,按你的范围选择。
- `.wechat-script-test/`:本地排版验证页、源码副本、截图和结果,需整理为可复现测试后再选择纳入。
- `legacy/`:默认不进入主链路首批提交;若保留历史实现,必须逐项排除帐号数据、虚拟环境、缓存及下载文章。
- `credits-video/`:滚动字幕视频工具,与本次流程无直接关系,默认不纳入本批。
### C. 建议排除的本地产物
真实帐号 CSV/JSON、下载文章及图片、下载历史数据库、真实发布记录与截图、文章包 `*.dww11.json`、浏览器配置和 Cookie、`.venv/`、`__pycache__/`、`*.pyc`、`.DS_Store`、视频输出和临时诊断文件。
首次提交准备时已增加根目录 `.gitignore`,覆盖真实帐号、运行记录、下载目录及本批范围之外的组件;按白名单添加源码和说明。
## 六、本轮验证结果
| 检查 | 结果 |
| --- | --- |
| Git 状态 | 当前目录不是 Git 仓库;本轮没有新建仓库 |
| 原始导入测试 `node --test test-dww11-importer.cjs` | 8 项中 3 项通过、5 项因旧素材目录不存在失败 |
| 只在内存中修正素材路径后执行同一测试 | 8/8 通过,覆盖打包与直接导入一致、缺图/长标题拦截、上传失败保留原文、富文本丢图回滚、无发表请求;磁盘测试文件未修改 |
| 下载器单元测试 | 默认 Python 3.15 与 Python 3.12 均因缺少 `requests` 无法加载测试;未安装依赖,不能认定通过或认定下载逻辑失败 |
| JavaScript 语法 | 6 个扩展 JS 按 ES 模块检查通过;4 个根目录油猴脚本检查通过 |
| Python 语法 | 打包器、下载器 app 与保存器均通过语法解析 |
| 扩展 ZIP | 11 个文件与源码目录逐一一致 |
| 在线验收 | 本轮未执行;历史文档中的单篇演示不等于当前版本全链路验收 |
## 七、首次提交整理范围
用户已授权新建本地仓库并完成 initial commit,范围为总流程文档、当前换号扩展、新闻下载器、文章打包/导入/排版及相关说明与测试。
- 增加根目录 README 与 `.gitignore`。
- 导入测试改用项目自有测试稿及三张程序生成的 PNG,修正打包说明路径。
- 原始流程说明备份到本地排除目录,再匿名化仓库文档中的帐号名称、UID 与电脑路径。
- 真实帐号、下载历史、演示 JSON、截图及原始说明均保留本地,不纳入提交。
- 下载器测试仍需先安装依赖,扩展缺失的测试文件尚未补齐;不以首次提交表示全流程已经完成。
- 本次只建立本地版本历史,未配置远程仓库或推送。