Watch
1
0
Fork
You've already forked packages
0

packages/ at markup-kit dd1e6eb5270e

A snapshot of packages/ in the Weft repository; the official packages are published from these commits.

Markup-Kit-Commit: dd1e6eb5270e62e0c8b6c202b3d01f0da6e00ddc
This commit is contained in:
Weft packages 2026-11-19 20:43:26 +00:00
commit 306b143104
72 changed files with 10915 additions and 0 deletions

202
LICENSE-APACHE Normal file
View file

@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

21
LICENSE-MIT Normal file
View file

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 The Weft authors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

48
README.md Normal file
View file

@ -0,0 +1,48 @@
# Official packages
Packages published under the `weft` owner, written in Weft over HTTP APIs and `std`. There is no npm bridge
(`docs/packages.md`, Rulings): anything an app needs from the JS ecosystem is written here instead.
| package | what | capabilities |
|---|---|---|
| [`weft/markdown`](markdown/README.md) | CommonMark and GFM: a syntax tree, HTML and a sanitiser | none |
| [`weft/stripe`](stripe/README.md) | Stripe's API: customers, Checkout, payments, subscriptions, prices, products, webhooks | `http.fetch@api.stripe.com` |
| [`weft/email`](email/README.md) | Resend and Postmark, MIME with attachments, templates rendered to HTML and text | `http.fetch@api.resend.com`, `http.fetch@api.postmarkapp.com`, `http.media` (its `Img` shows the image a template names) |
Each folder is a Weft project with its own `weft.toml`. Its tests, benchmarks and scripts stay out of the published archive
(`include` in the manifest).
## Trying them against a local registry
`scripts/verify-local.mjs` runs the whole path on this machine: it starts `apps/registry`, makes a publish token and a
signing key for the owner `weft`, publishes each package signed, and then in a fresh app runs `weft add`, `check`, `run`,
`build --target js` and `install --frozen`.
```sh
cargo build --release -p weft-cli # in core/
node packages/scripts/verify-local.mjs # every package, or name some: markdown stripe email
```
## Publishing for real
None of this has happened yet. In order:
1. **Deploy the registry** at pkg.weft-lang.dev (`apps/registry/README.md`, What a deploy needs).
2. **Make the owner's token**: `POST /v1/tokens` with `{ "owners": ["weft"] }` and the operator's `REGISTRY_ADMIN_TOKEN`. `weft` is
a reserved owner, so only the operator can grant it.
3. **Make the signing key offline**: `weft publish --new-key`, keep the secret out of the repository, and register the public
key with `PUT /v1/owners/weft/key`. From then on the registry takes only versions signed by it.
4. **Set each manifest's `repository`** to the public source. It is a placeholder today
(`https://git.weft-lang.dev/weft/packages`), and a signed statement records it for good.
5. **Publish from a clean checkout**, with the commit recorded: `WEFT_TOKEN=… WEFT_SIGNING_KEY=… WEFT_COMMIT=$(git rev-parse HEAD)
weft publish --sign --dir packages/<name>`.
6. **Look at the generated pages** (`/packages/weft/<name>`) before announcing a version: published versions can be yanked but
never changed.
Before each package's first real release:
| package | still to do |
|---|---|
| `weft/markdown` | nothing beyond the steps above |
| `weft/stripe` | run its calls against a Stripe test-mode key: so far they have met only stripe-mock and a scripted server |
| `weft/email` | send a message through Resend and Postmark with test keys: so far they have met only a scripted server |

132
email/README.md Normal file
View file

@ -0,0 +1,132 @@
# @weft/email
Email for Weft: sending through Resend and Postmark, MIME messages with attachments, and templates that render to HTML and
text. Its only capabilities are `http.fetch@api.resend.com`, `http.fetch@api.postmarkapp.com` and `http.media`, because its
`Img` component shows the image a template names.
```sh
weft add weft/email
```
```weft
import { send } from "@weft/email"
import { resend } from "@weft/email/resend"
import { document } from "@weft/email/template"
import { render } from "std/app"
import { env } from "std/process"
import Receipt from "./Receipt"
let mail = resend({ apiKey: env("RESEND_API_KEY")! })
let body = document(render(Receipt, { name: "Ana", total: "19,99 €" }), {
title: "Your receipt",
preheader: "Thanks for your order",
footer: "Example d.o.o., Ljubljana",
})
let sent = try await send(mail, {
from: { email: "info@example.com", name: "Example" },
to: ["ana@example.com"],
subject: "Your receipt",
html: body.html,
text: body.text,
})
```
```weft file=Receipt.weft
import Button from "@weft/email/components/Button"
import Facts from "@weft/email/components/Facts"
import H1 from "@weft/email/components/H1"
import P from "@weft/email/components/P"
import Strong from "@weft/email/components/Strong"
prop name: string
prop total: string
---
<H1>Thank you!</H1>
<P>Hi <Strong>{name}</Strong>, your payment came through.</P>
<Facts rows={[["Total", total]]} />
<Button url="https://example.com/orders" label="Your orders" />
```
## Modules
| module | what |
|---|---|
| `@weft/email` | `Message`, `send`, `validate`, `allowlist`, `memory` and `outbox` for tests |
| `@weft/email/resend` | `resend({ apiKey })` |
| `@weft/email/postmark` | `postmark({ serverToken, messageStream? })` |
| `@weft/email/mime` | `toMime(message)`, the message as RFC 5322 text |
| `@weft/email/template` | `document`, `isSafeUrl` |
| `@weft/email/components/…` | `H1`, `H2`, `P`, `Strong`, `Em`, `A`, `Img`, `Ul`, `Ol`, `Facts`, `Button`, `Hr` |
## Messages
`send` validates a message before anything leaves. A line break in a subject, a display name or a header is refused, because it
would let whoever typed that text add headers of their own, such as a `Bcc`.
```weft
try await send(mail, {
from: { email: "info@example.com", name: "Example" },
to: ["ana@example.com"],
replyTo: "owner@example.com",
subject: "Your ticket",
html: body.html,
text: body.text,
headers: [["List-Unsubscribe", "<https://example.com/unsubscribe?id=1>"]],
attachments: [
{ filename: "ticket.pdf", content: pdf, type: "application/pdf" },
{ filename: "logo.png", content: logo, type: "image/png", contentId: "logo" }, // <img src="cid:logo">
],
})
```
## Errors are values
| `EmailError` | what to do |
|---|---|
| `InvalidMessage { field, message }` | fix the message: nothing was attempted |
| `Rejected { status, code, message }` | a bad key, an unverified domain, a suppressed address: sending it again won't help |
| `RateLimited { message, retryAfter }` | try again later |
| `ProviderFailure { status, message }` | try again later |
| `ConnectionFailed { message }` | the message **may** have been sent |
| `BadResponse { message }` | the message may have been sent |
Nothing is retried for you, because sending an email twice is worse than a delay. With Resend, `idempotencyKey` makes a repeat
safe:
```weft
let mail = resend({ apiKey, idempotencyKey: m => `order-${orderId}` })
```
## Staging
`allowlist` sends only to the addresses listed, and skips the rest without failing, so a staging site can't email real customers.
`"*"` lifts the limit:
```weft
let mail = allowlist(resend({ apiKey }), (env("EMAIL_ALLOWLIST") ?? "info@example.com").split(","))
```
In tests, `memory()` keeps messages in `outbox()` instead of sending them.
## Templates
A template is a Weft component. `render` from `std/app` turns it into HTML with inline styles and a text version that reads as if a
person wrote it. Everything a component shows is escaped, and a link that isn't `http`, `https`, `mailto`, `tel` or `cid` keeps
only its text. Your own markup works too: classes become inline styles, and `hover:`, `dark:` and breakpoint classes are left out.
`document` wraps a rendered template in the whole email, a centred column that works in mail clients.
## What has and hasn't been tested
`node packages/email/scripts/integration.mjs` sends through both providers' APIs to a scripted server, and checks a full message
with Python's `email` parser, an independent reader of the MIME. `weft test tests/email.test.weft` covers validation, encoding,
payloads and templates. **No email has been sent through Resend or Postmark themselves**: the payloads follow their API references,
so try a test key before relying on this.
## Limits
- **No SMTP.** `std` has no TCP or TLS sockets, so this package can't talk to a mail server. `toMime` already writes what SMTP's
`DATA` needs, so only the socket is missing.
- **It reaches only Resend's and Postmark's API hosts.** As a dependency, a `baseUrl` on any other host fails with a
`CapabilityError`. To point it at a test server, the app lists that host in `allow` too, beside the two API hosts.

8
email/components/A.weft Normal file
View file

@ -0,0 +1,8 @@
/// A link. An address that isn't `http`, `https`, `mailto`, `tel` or `cid` renders as its text.
import { isSafeUrl } from "../template"
prop href: string
prop children: Snippet
---
@if isSafeUrl(href) {<Link
to={Route.url(href)}
class="underline">@render children()</Link>} @else {@render children()}

View file

@ -0,0 +1,18 @@
/// A button. In the text version it is its label and its address.
import type { Color } from "std/color"
import { isSafeUrl } from "../template"
prop url: string
prop label: string
prop color: Color = "#111111"
prop textColor: Color = "#ffffff"
---
@if isSafeUrl(url) {
<Box class="my-5">
<Box class="inline-block rounded-md" style={{ background: color }}>
<Link to={Route.url(url)} class="inline-block px-5 py-3 no-underline font-semibold"><Text
style={{ color: textColor }}>{label}</Text></Link>
</Box>
</Box>
} @else {
<Box class="mb-4">{label}</Box>
}

4
email/components/Em.weft Normal file
View file

@ -0,0 +1,4 @@
/// Italic text.
prop children: Snippet
---
<Box class="inline italic">@render children()</Box>

View file

@ -0,0 +1,9 @@
/// Labels and values, as on a receipt.
prop rows: [string, string][]
---
<Box role="list" class="mb-4">
@for row of rows {
<Box class="flex gap-3 py-1"><Text
class="text-muted-foreground">{row[0]}:</Text> <Text>{row[1]}</Text></Box>
}
</Box>

4
email/components/H1.weft Normal file
View file

@ -0,0 +1,4 @@
/// The largest heading.
prop children: Snippet
---
<Box class="mb-4 text-2xl font-bold">@render children()</Box>

4
email/components/H2.weft Normal file
View file

@ -0,0 +1,4 @@
/// A heading.
prop children: Snippet
---
<Box class="mb-3 text-lg font-bold">@render children()</Box>

3
email/components/Hr.weft Normal file
View file

@ -0,0 +1,3 @@
/// A horizontal rule.
---
<Box role="separator" class="my-5 border-t border-border" />

13
email/components/Img.weft Normal file
View file

@ -0,0 +1,13 @@
/// An image. Use an absolute URL, or `cid:<contentId>` for an attachment sent inline.
import { isSafeUrl } from "../template"
prop src: string
prop alt: string
prop width?: float
---
@if !isSafeUrl(src) {
<Text>{alt}</Text>
} @else if width != null {
<Image src={src} label={alt} class="max-w-full h-auto" style={{ width }} />
} @else {
<Image src={src} label={alt} class="max-w-full h-auto" />
}

8
email/components/Ol.weft Normal file
View file

@ -0,0 +1,8 @@
/// A numbered list, one line for each item.
prop items: string[]
---
<Box role="list" class="mb-4">
@for [i, item] of items.entries() {
<Box class="mb-1">{i + 1}. {item}</Box>
}
</Box>

4
email/components/P.weft Normal file
View file

@ -0,0 +1,4 @@
/// A paragraph.
prop children: Snippet
---
<Box class="mb-4">@render children()</Box>

View file

@ -0,0 +1,4 @@
/// Bold text.
prop children: Snippet
---
<Box class="inline font-bold">@render children()</Box>

8
email/components/Ul.weft Normal file
View file

@ -0,0 +1,8 @@
/// A list, one line for each item.
prop items: string[]
---
<Box role="list" class="mb-4">
@for item of items {
<Box class="mb-1">- {item}</Box>
}
</Box>

227
email/email.weft Normal file
View file

@ -0,0 +1,227 @@
// `@weft/email`: messages, their validation and sending them through a transport.
/// An email address, with or without a display name: `"ana@example.com"` or `{ email: "ana@example.com", name: "Ana" }`.
export type Address = string | { email: string, name?: string }
/// A file sent with a message. `type` is its media type (`application/octet-stream` unless given). `contentId` makes it an
/// inline image, which the HTML shows with `<img src="cid:…">`.
export type Attachment = { filename: string, content: Bytes, type?: string, contentId?: string }
/// A message. It needs a `from`, at least one recipient, a `subject` and an `html` or `text` body; send both when you can,
/// since spam filters and some readers want the text. `headers` are extra headers, such as `List-Unsubscribe`.
export type Message = {
from: Address,
to: Address[],
cc?: Address[],
bcc?: Address[],
replyTo?: Address,
subject: string,
html?: string,
text?: string,
headers?: [string, string][],
attachments?: Attachment[],
}
/// What sending gave: the provider's id for the message. `skipped` is true when `allowlist` kept it from being sent.
export type Sent = { id: string, provider: string, skipped: boolean }
/// Why a message wasn't sent.
///
/// - `InvalidMessage`: the message can't be sent as it is, and nothing was attempted. `field` says what is wrong.
/// - `Rejected`: the provider refused it (a bad key, an unverified domain, a suppressed recipient). Sending it again won't help.
/// - `RateLimited`: too many messages. Try again later.
/// - `ProviderFailure`: the provider had a problem. Try again later.
/// - `ConnectionFailed`: the provider wasn't reached, or its answer broke off, so the message **may** have been sent.
/// - `BadResponse`: the provider's answer couldn't be read, so the message may have been sent.
export type EmailError =
| InvalidMessage { field: string, message: string }
| Rejected { status: float, code: string?, message: string }
| RateLimited { message: string, retryAfter: float? }
| ProviderFailure { status: float, message: string }
| ConnectionFailed { message: string }
| BadResponse { message: string }
/// How messages leave: `resend(…)`, `postmark(…)`, `memory()`, or your own.
export type Transport = { name: string, send: (message: Message) => Task<Sent ! EmailError> }
/// The plain address of an `Address`.
export fn emailOf(address: Address): string {
return address is string ? address : address.email
}
/// The display name of an `Address`, if it has one.
export fn nameOf(address: Address): string? {
return address is string ? null : address.name
}
fn hasLineBreak(s: string): boolean {
return s.includes("\r") || s.includes("\n")
}
fn isAddressChar(c: int): boolean {
// Printable ASCII but what would end an address in a header, or anything from outside ASCII (internationalised mail).
return c > 32 &&
c != 127 &&
c != 60 &&
c != 62 &&
c != 40 &&
c != 41 &&
c != 44 &&
c != 59 &&
c != 58 &&
c != 34 &&
c != 92 &&
c != 91 &&
c != 93
}
/// Whether `email` has the shape of an address: one `@`, something on both sides, nothing that would end it in a header.
export fn isValidEmail(email: string): boolean {
let at = email.indexOf("@")
if at <= 0 || at != email.lastIndexOf("@") || at == email.length - 1 || email.length > 254 {
return false
}
for i of 0..email.length {
let c = email.codeUnitAt(i) ?? 0
if c != 64 && !isAddressChar(c) {
return false
}
}
let domain = email.slice(at + 1)
return !domain.startsWith(".") && !domain.endsWith(".") && !domain.includes("..")
}
fn checkAddress(field: string, address: Address): void ! EmailError {
let email = emailOf(address)
if !isValidEmail(email) {
fail InvalidMessage { field, message: `\`${email}\` isn't an email address` }
}
if hasLineBreak(nameOf(address) ?? "") {
fail InvalidMessage { field, message: "a display name can't hold a line break" }
}
}
fn isHeaderName(name: string): boolean {
if name == "" {
return false
}
for i of 0..name.length {
let c = name.codeUnitAt(i) ?? 0
if c <= 32 || c >= 127 || c == 58 {
return false
}
}
return true
}
/// Checks a message before it is sent. Line breaks in a subject, a name or a header are refused, because they would let
/// whoever supplied that text add headers of their own.
export fn validate(message: Message): void ! EmailError {
try checkAddress("from", message.from)
let to = message.to
let cc = message.cc ?? []
let bcc = message.bcc ?? []
if to.length + cc.length + bcc.length == 0 {
fail InvalidMessage { field: "to", message: "a message needs at least one recipient" }
}
for address of to {
try checkAddress("to", address)
}
for address of cc {
try checkAddress("cc", address)
}
for address of bcc {
try checkAddress("bcc", address)
}
let replyTo = message.replyTo
if replyTo != null {
try checkAddress("replyTo", replyTo)
}
if message.subject.trim() == "" {
fail InvalidMessage { field: "subject", message: "a message needs a subject" }
}
if hasLineBreak(message.subject) {
fail InvalidMessage { field: "subject", message: "a subject can't hold a line break" }
}
if (message.html ?? "") == "" && (message.text ?? "") == "" {
fail InvalidMessage { field: "html", message: "a message needs an html or a text body" }
}
for [name, value] of message.headers ?? [] {
if !isHeaderName(name) {
fail InvalidMessage { field: "headers", message: `\`${name}\` isn't a header name` }
}
if hasLineBreak(value) {
fail InvalidMessage {
field: "headers",
message: `the header \`${name}\` can't hold a line break`,
}
}
}
for attachment of message.attachments ?? [] {
if attachment.filename == "" || hasLineBreak(attachment.filename) {
fail InvalidMessage {
field: "attachments",
message: "an attachment needs a file name on one line",
}
}
}
}
/// Checks a message and sends it.
///
/// ```weft
/// import { send } from "@weft/email"
/// import { resend } from "@weft/email/resend"
///
/// let mail = resend({ apiKey: env("RESEND_API_KEY")! })
/// let sent = try await send(mail, {
/// from: { email: "info@example.com", name: "Example" },
/// to: ["ana@example.com"],
/// subject: "Your ticket",
/// html: "<p>See you there.</p>",
/// text: "See you there.",
/// })
/// ```
export async fn send(transport: Transport, message: Message): Sent ! EmailError {
try validate(message)
return try await transport.send(message)
}
/// `inner`, sending only to the addresses listed (`"*"` lifts the limit). Other messages are skipped, not failed: staging
/// sites use this so that real customers never get a test email.
export fn allowlist(inner: Transport, allowed: string[]): Transport {
let list = allowed.map(a => a.trim().toLowerCase())
let open = list.includes("*")
return {
name: inner.name,
send: async message => {
let all = [...message.to, ...(message.cc ?? []), ...(message.bcc ?? [])]
if !open && !all.every(a => list.includes(emailOf(a).trim().toLowerCase())) {
return { id: "", provider: inner.name, skipped: true }
}
return try await inner.send(message)
},
}
}
let mut sentMessages: Message[] = []
/// A transport that keeps messages in this process instead of sending them, for tests. Read them with `outbox()`.
export fn memory(): Transport {
return {
name: "memory",
send: async message => {
sentMessages.push(message)
return { id: `memory-${sentMessages.length}`, provider: "memory", skipped: false }
},
}
}
/// What `memory()` has been given, oldest first.
export fn outbox(): Message[] {
return sentMessages
}
/// Empties `outbox()`.
export fn clearOutbox() {
sentMessages = []
}

208
email/mime.weft Normal file
View file

@ -0,0 +1,208 @@
// MIME: a message as the text mail servers exchange (RFC 5322, RFC 2045–2047, RFC 2231).
import { uuid } from "std/random"
import { emailOf, nameOf } from "./email"
import type { Address, Attachment, Message } from "./email"
fn isAscii(s: string): boolean {
for i of 0..s.length {
let c = s.codeUnitAt(i) ?? 0
if c >= 127 || c < 32 {
return false
}
}
return true
}
/// `text` for a header: as it is when it is plain ASCII, otherwise as RFC 2047 encoded words of at most 75 characters,
/// never splitting a character.
export fn encodeHeader(text: string): string {
if isAscii(text) {
return text
}
let mut words: string[] = []
let mut chunk: int[] = []
for ch of text {
let bytes = Bytes.fromUtf8(ch).toList()
// 45 bytes of base64 fill an encoded word's 60 characters.
if chunk.length + bytes.length > 45 {
words.push(`=?UTF-8?B?${Bytes.of(chunk).toBase64()}?=`)
chunk = []
}
chunk.push(...bytes)
}
if chunk.length > 0 {
words.push(`=?UTF-8?B?${Bytes.of(chunk).toBase64()}?=`)
}
return words.join("\r\n ")
}
fn quoted(name: string): string {
return `"${name.replaceAll("\\", "\\\\").replaceAll("\"", "\\\"")}"`
}
/// An address as a header writes it.
export fn formatAddress(address: Address): string {
let email = emailOf(address)
let name = nameOf(address)
if name == null || name == "" {
return email
}
return isAscii(name) ? `${quoted(name)} <${email}>` : `${encodeHeader(name)} <${email}>`
}
fn addressList(list: Address[]): string {
return list.map(formatAddress).join(",\r\n ")
}
/// One character of a line in quoted-printable. Spaces and tabs at the end of a line are encoded, as RFC 2045 requires.
fn encodeChar(ch: string, last: boolean): string {
let c = ch.codePointAt(0) ?? 0
if (c >= 33 && c <= 126 && c != 61) || ((c == 32 || c == 9) && !last) {
return ch
}
return Bytes.fromUtf8(ch)
.toList()
.map(b => "=" + str(b, { radix: 16 }).toUpperCase().padStart(2, "0"))
.join("")
}
/// `text` in quoted-printable (RFC 2045): lines of at most 76 characters, with no character split across two of them.
export fn quotedPrintable(text: string): string {
let source = text.replaceAll("\r\n", "\n").replaceAll("\r", "\n")
let mut lines: string[] = []
for line of source.split("\n") {
let mut all: string[] = []
for ch of line {
all.push(ch)
}
let mut out: string[] = []
let mut width = 0
for [i, ch] of all.entries() {
let piece = encodeChar(ch, i == all.length - 1)
if width + piece.length > 75 {
out.push("=\r\n")
width = 0
}
out.push(piece)
width += piece.length
}
lines.push(out.join(""))
}
return lines.join("\r\n")
}
fn wrapBase64(bytes: Bytes): string {
let text = bytes.toBase64()
let mut lines: string[] = []
let mut i = 0
while i < text.length {
lines.push(text.slice(i, i + 76))
i += 76
}
return lines.join("\r\n")
}
fn textPart(media: string, body: string): string {
return `Content-Type: ${media}; charset=UTF-8\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n${quotedPrintable(body)}`
}
fn quotedParam(value: string): string {
return quoted(value)
}
fn percentEncoded(text: string): string {
let mut out: string[] = []
for b of Bytes.fromUtf8(text).toList() {
let plain = (b >= 48 && b <= 57) ||
(b >= 65 && b <= 90) ||
(b >= 97 && b <= 122) ||
b == 45 ||
b == 46 ||
b == 95
out.push(plain ? String.fromCharCode(b) : "%" + str(b, { radix: 16 }).toUpperCase().padStart(2, "0"))
}
return out.join("")
}
fn fileParam(name: string, filename: string): string {
return isAscii(filename)
? `${name}=${quotedParam(filename)}`
: `${name}*=UTF-8''${percentEncoded(filename)}`
}
fn attachmentPart(a: Attachment): string {
let media = a.type ?? "application/octet-stream"
let inline = a.contentId != null
let head = [
`Content-Type: ${media}; ${fileParam("name", a.filename)}`,
"Content-Transfer-Encoding: base64",
`Content-Disposition: ${inline ? "inline" : "attachment"}; ${fileParam("filename", a.filename)}`,
]
let cid = a.contentId
let lines = cid != null ? [...head, `Content-ID: <${cid}>`] : head
return `${lines.join("\r\n")}\r\n\r\n${wrapBase64(a.content)}`
}
fn multipart(subtype: string, boundary: string, parts: string[]): string {
let body = parts.map(p => `--${boundary}\r\n${p}\r\n`).join("")
return `Content-Type: multipart/${subtype};\r\n boundary="${boundary}"\r\n\r\n${body}--${boundary}--`
}
/// `id` makes the boundaries and the `Message-ID` repeatable, and `date` is the `Date` header (RFC 5322 form), for tests.
export type MimeOptions = { id?: string, date?: string }
fn domainOf(address: Address): string {
let email = emailOf(address)
return email.slice(email.indexOf("@") + 1)
}
/// The whole message as MIME text, with CRLF line endings and only ASCII in it: what an SMTP server is given after `DATA`.
/// `bcc` isn't written, since every recipient would see it.
///
/// ```weft
/// let source = toMime({ from: "a@example.com", to: ["b@example.com"], subject: "Hi", text: "Hello" })
/// ```
export fn toMime(message: Message, options: MimeOptions = {}): string {
let id = options.id ?? uuid()
let mut headers: string[] = [
`From: ${formatAddress(message.from)}`,
`To: ${addressList(message.to)}`,
]
let cc = message.cc ?? []
if cc.length > 0 {
headers.push(`Cc: ${addressList(cc)}`)
}
let replyTo = message.replyTo
if replyTo != null {
headers.push(`Reply-To: ${formatAddress(replyTo)}`)
}
headers.push(`Subject: ${encodeHeader(message.subject)}`)
let date = options.date
if date != null {
headers.push(`Date: ${date}`)
}
headers.push(`Message-ID: <${id}@${domainOf(message.from)}>`)
headers.push("MIME-Version: 1.0")
for [name, value] of message.headers ?? [] {
headers.push(`${name}: ${encodeHeader(value)}`)
}
let text = message.text ?? ""
let html = message.html ?? ""
let mut body = text != "" && html != ""
? multipart(
"alternative",
`alt-${id}`,
[textPart("text/plain", text), textPart("text/html", html)],
)
: html != "" ? textPart("text/html", html) : textPart("text/plain", text)
let attachments = message.attachments ?? []
let inline = attachments.filter(a => a.contentId != null)
let files = attachments.filter(a => a.contentId == null)
if inline.length > 0 {
body = multipart("related", `rel-${id}`, [body, ...inline.map(attachmentPart)])
}
if files.length > 0 {
body = multipart("mixed", `mix-${id}`, [body, ...files.map(attachmentPart)])
}
return `${headers.join("\r\n")}\r\n${body}\r\n`
}

131
email/postmark.weft Normal file
View file

@ -0,0 +1,131 @@
// Sending through Postmark (https://postmarkapp.com/developer/api/email-api).
import { fetch } from "std/http"
import type { Address, EmailError, Message, Transport } from "./email"
import { formatAddress } from "./mime"
/// `messageStream` is Postmark's stream to send on (`outbound` unless given; broadcasts need `broadcast`). `baseUrl` is for tests.
export type PostmarkOptions = {
serverToken: string,
messageStream?: string,
baseUrl?: string,
timeout?: float,
}
fn text(value: json): string? {
return value is string ? value : null
}
fn list(addresses: Address[]): string {
return addresses.map(formatAddress).join(", ")
}
/// The JSON Postmark is sent for a message.
export fn payload(message: Message, messageStream: string = "outbound"): json {
let mut body: json = {
From: formatAddress(message.from),
To: list(message.to),
Subject: message.subject,
MessageStream: messageStream,
}
let cc = message.cc ?? []
if cc.length > 0 {
body["Cc"] = list(cc)
}
let bcc = message.bcc ?? []
if bcc.length > 0 {
body["Bcc"] = list(bcc)
}
let replyTo = message.replyTo
if replyTo != null {
body["ReplyTo"] = formatAddress(replyTo)
}
let html = message.html
if html != null && html != "" {
body["HtmlBody"] = html
}
let plainText = message.text
if plainText != null && plainText != "" {
body["TextBody"] = plainText
}
let headers = message.headers ?? []
if headers.length > 0 {
let fields: json[] = headers.map(h => {
let field: json = { Name: h[0], Value: h[1] }
return field
})
body["Headers"] = fields
}
let attachments = message.attachments ?? []
if attachments.length > 0 {
let files: json[] = attachments.map(a => {
let mut file: json = {
Name: a.filename,
Content: a.content.toBase64(),
ContentType: a.type ?? "application/octet-stream",
}
let cid = a.contentId
if cid != null {
file["ContentID"] = `cid:${cid}`
}
return file
})
body["Attachments"] = files
}
return body
}
/// The error for an answer that isn't a success.
export fn failureOf(status: float, body: json): EmailError {
let message = text(body.Message) ?? `Postmark answered ${status}`
let code = body.ErrorCode
if status == 429 {
return RateLimited { message, retryAfter: null }
}
if status >= 500 {
return ProviderFailure { status, message }
}
return Rejected { status, code: code is float ? `${code}` : null, message }
}
/// A transport that sends through Postmark. It needs `http.fetch`.
///
/// ```weft
/// let mail = postmark({ serverToken: env("POSTMARK_TOKEN")! })
/// ```
export fn postmark(options: PostmarkOptions): Transport {
let url = `${options.baseUrl ?? "https://api.postmarkapp.com"}/email`
let timeout = options.timeout ?? 30000
let token = options.serverToken
let stream = options.messageStream ?? "outbound"
return {
name: "postmark",
send: async message => {
let headers: [string, string][] = [
["x-postmark-server-token", token],
["accept", "application/json"],
["user-agent", "weft-email/0.1.0"],
]
let res, err = try await fetch(
url,
{ method: "POST", headers, json: payload(message, stream), timeout },
)
if err != null {
fail ConnectionFailed { message: `could not reach Postmark: ${err.message}` }
}
let answer = try await res.json() else null
if !res.ok {
fail failureOf(res.status, answer)
}
// Postmark can answer 200 with an error code of its own.
let code = answer.ErrorCode
if code is float && code != 0 {
fail failureOf(422, answer)
}
let id = text(answer.MessageID)
if id == null {
fail BadResponse { message: "Postmark's answer has no MessageID" }
}
return { id, provider: "postmark", skipped: false }
},
}
}

146
email/resend.weft Normal file
View file

@ -0,0 +1,146 @@
// Sending through Resend (https://resend.com/docs/api-reference/emails/send-email).
import { fetch } from "std/http"
import { emailOf } from "./email"
import type { Address, EmailError, Message, Transport } from "./email"
/// `baseUrl` is for tests. `idempotencyKey` is sent with every message when given as a function of the message, so that
/// sending one again by accident does nothing.
export type ResendOptions = {
apiKey: string,
baseUrl?: string,
timeout?: float,
idempotencyKey?: (message: Message) => string?,
}
fn text(value: json): string? {
return value is string ? value : null
}
fn plain(address: Address): string {
// Resend reads a UTF-8 display name directly, so it isn't encoded as a mail header would be.
let email = emailOf(address)
let name = address is string ? null : address.name
if name == null || name == "" {
return email
}
return `"${name.replaceAll("\\", "\\\\").replaceAll("\"", "\\\"")}" <${email}>`
}
/// The JSON Resend is sent for a message.
export fn payload(message: Message): json {
let mut body: json = {
from: plain(message.from),
to: message.to.map(plain),
subject: message.subject,
}
let cc = message.cc ?? []
if cc.length > 0 {
body["cc"] = cc.map(plain)
}
let bcc = message.bcc ?? []
if bcc.length > 0 {
body["bcc"] = bcc.map(plain)
}
let replyTo = message.replyTo
if replyTo != null {
body["reply_to"] = plain(replyTo)
}
let html = message.html
if html != null && html != "" {
body["html"] = html
}
let plainText = message.text
if plainText != null && plainText != "" {
body["text"] = plainText
}
let headers = message.headers ?? []
if headers.length > 0 {
body["headers"] = Map<string, string>(headers)
}
let attachments = message.attachments ?? []
if attachments.length > 0 {
let files: json[] = attachments.map(a => {
let mut file: json = { filename: a.filename, content: a.content.toBase64() }
let media = a.type
if media != null {
file["content_type"] = media
}
let cid = a.contentId
if cid != null {
file["content_id"] = cid
}
return file
})
body["attachments"] = files
}
return body
}
/// The error for an answer that isn't a success.
export fn failureOf(status: float, body: json, retryAfter: string?): EmailError {
let message = text(body.message) ?? `Resend answered ${status}`
if status == 429 {
return RateLimited { message, retryAfter: retryAfter != null ? secondsOf(retryAfter) : null }
}
if status >= 500 {
return ProviderFailure { status, message }
}
return Rejected { status, code: text(body.name), message }
}
fn secondsOf(value: string): float? {
let n = value.trim()
let mut total = 0
if n == "" {
return null
}
for ch of n {
let c = ch.codeUnitAt(0) ?? 0
if c < 48 || c > 57 {
return null
}
total = total * 10 + (c - 48)
}
return total
}
/// A transport that sends through Resend. It needs `http.fetch`.
///
/// ```weft
/// let mail = resend({ apiKey: env("RESEND_API_KEY")! })
/// ```
export fn resend(options: ResendOptions): Transport {
let url = `${options.baseUrl ?? "https://api.resend.com"}/emails`
let timeout = options.timeout ?? 30000
let keyOf = options.idempotencyKey
let apiKey = options.apiKey
return {
name: "resend",
send: async message => {
let mut headers: [string, string][] = [
["authorization", `Bearer ${apiKey}`],
["user-agent", "weft-email/0.1.0"],
]
let key = keyOf != null ? keyOf(message) : null
if key != null {
headers.push(["idempotency-key", key])
}
let res, err = try await fetch(
url,
{ method: "POST", headers, json: payload(message), timeout },
)
if err != null {
fail ConnectionFailed { message: `could not reach Resend: ${err.message}` }
}
let answer = try await res.json() else null
if !res.ok {
fail failureOf(res.status, answer, res.headers.get("retry-after"))
}
let id = text(answer.id)
if id == null {
fail BadResponse { message: "Resend's answer has no id" }
}
return { id, provider: "resend", skipped: false }
},
}
}

View file

@ -0,0 +1,49 @@
# Reads the message packages/integration/email prints and checks it with Python's own `email` parser: an independent reader
# of what `toMime` writes. Usage: python3 check_mime.py <file with the message>
import email
import email.policy
import sys
raw = open(sys.argv[1], "rb").read()
assert b"\r\n" in raw and b"\n" not in raw.replace(b"\r\n", b""), "every line ends in CRLF"
assert all(b < 128 for b in raw), "the message is ASCII"
for line in raw.split(b"\r\n"):
assert len(line) <= 998, "no line is over RFC 5322's limit"
msg = email.message_from_bytes(raw, policy=email.policy.default)
defects = list(msg.defects) + [d for part in msg.walk() for d in part.defects]
assert not defects, f"the parser found defects: {defects}"
assert msg["Subject"] == "Vaša vstopnica za dogodek: čudovit večer na gradu, ki se ga boste še dolgo spominjali", msg["Subject"]
assert msg["From"].addresses[0].display_name == 'Žiga "Vidni" Novak', msg["From"]
assert msg["From"].addresses[0].addr_spec == "info@example.com"
to = msg["To"].addresses
assert [a.addr_spec for a in to] == ["ana@example.com", "bor@example.com"], msg["To"]
assert to[1].display_name == "Bor Šuštar", to[1].display_name
assert msg["Cc"].addresses[0].addr_spec == "cene@example.com"
assert msg["Bcc"] is None and b"skrito@example.com" not in raw, "bcc isn't written"
assert msg["Reply-To"].addresses[0].addr_spec == "lastnik@example.com"
assert msg["List-Unsubscribe"] == "<https://example.com/odjava?id=1>"
assert msg["X-Oznaka"] == "račun", msg["X-Oznaka"]
assert msg["MIME-Version"] == "1.0"
assert msg["Message-ID"].endswith("@example.com>")
assert msg.get_content_type() == "multipart/mixed", msg.get_content_type()
long = "Pozdravljeni, to je dolgo sporočilo s črkami č, š in ž, ki sega čez mejo šestinsedemdesetih znakov v vrstici. = in pika.\n.vrstica s piko\nKonec "
plain = msg.get_body(preferencelist=("plain",)).get_content()
assert plain.replace("\r\n", "\n").rstrip("\n") == long, repr(plain)
html = msg.get_body(preferencelist=("html",)).get_content()
assert html.replace("\r\n", "\n").rstrip("\n") == f'<p>{long}</p><img src="cid:logo">', repr(html)
files = {part.get_filename(): part for part in msg.iter_attachments()}
assert set(files) == {"račun št. 5.pdf", "dogodek.ics"}, set(files)
assert files["račun št. 5.pdf"].get_content() == bytes(range(256))
assert files["račun št. 5.pdf"].get_content_type() == "application/pdf"
assert files["dogodek.ics"].get_content_type() == "text/calendar"
inline = [p for p in msg.walk() if p.get("Content-ID")]
assert len(inline) == 1 and inline[0]["Content-ID"] == "<logo>" and inline[0].get_content_disposition() == "inline"
assert inline[0].get_payload(decode=True) == bytes([137, 80, 78, 71])
kinds = [p.get_content_type() for p in msg.walk()]
assert kinds[:4] == ["multipart/mixed", "multipart/related", "multipart/alternative", "text/plain"], kinds
print("Python's email parser reads the message back exactly")

View file

@ -0,0 +1,28 @@
// Runs packages/integration/email, then checks the MIME it prints with Python's `email` parser.
import { spawnSync } from "node:child_process"
import { mkdirSync, writeFileSync } from "node:fs"
import { dirname, join } from "node:path"
import { fileURLToPath } from "node:url"
const root = join(dirname(fileURLToPath(import.meta.url)), "..")
const weft = process.env.WEFT_BIN ?? join(root, "../../core/target/release/weft")
const app = join(root, "../integration/email")
spawnSync(weft, ["install"], { cwd: app, stdio: "inherit" })
const r = spawnSync(weft, ["run", "."], { cwd: app, encoding: "utf8", env: { ...process.env, WEFT_DEADLINE_MS: "0" } })
const out = r.stdout
const [before, rest] = out.split("=====MIME=====\n")
if (!rest) {
console.log(out, r.stderr)
process.exit(1)
}
const [mime, after] = rest.split("=====END=====\n")
process.stdout.write(before + after)
if (r.stderr) process.stderr.write(r.stderr)
const dir = join(root, "../../core/target/tools")
mkdirSync(dir, { recursive: true })
const file = join(dir, "email-sample.eml")
writeFileSync(file, mime, "latin1")
const py = spawnSync("python3", ["-I", join(root, "scripts/check_mime.py"), file], { encoding: "utf8" })
process.stdout.write(py.stdout)
process.stderr.write(py.stderr)
process.exit(r.status === 0 && py.status === 0 ? 0 : 1)

45
email/template.weft Normal file
View file

@ -0,0 +1,45 @@
// Email templates are Weft components: write one with the components in `components/`, render it with `render` from `std/app`,
// and give the result to `document`. Everything shown in a component is escaped, so values from users are safe in it.
/// An email rendered both ways: what `render` from `std/app` gives.
export type Rendered = { html: string, text: string }
fn escape(s: string): string {
return s.replaceAll("&", "&amp;")
.replaceAll("<", "&lt;")
.replaceAll(">", "&gt;")
.replaceAll("\"", "&quot;")
}
/// Whether an address may go in a link or an image: `http`, `https`, `mailto`, `tel` or `cid`.
export fn isSafeUrl(url: string): boolean {
let lower = url.trim().toLowerCase()
return lower.startsWith("http://") ||
lower.startsWith("https://") ||
lower.startsWith("mailto:") ||
lower.startsWith("tel:") ||
lower.startsWith("cid:")
}
/// What surrounds an email's content. `preheader` is the line inboxes show beside the subject; `footer` is the small print;
/// `lang` is the language of the text (`en`).
export type DocumentOptions = { title?: string, preheader?: string, footer?: string, lang?: string }
/// A rendered template as a whole email: its content in a centred column, with the preheader and footer.
///
/// ```weft
/// let mail = document(render(Receipt, { name: "Ana", total: "19,99 €" }), { title: "Your receipt", footer: "Example d.o.o." })
/// ```
export fn document(content: Rendered, options: DocumentOptions = {}): Rendered {
let body = content.html
let preheader = options.preheader ?? ""
let hidden = preheader != ""
? `<div style="display:none;max-height:0;overflow:hidden;opacity:0">${escape(preheader)}</div>`
: ""
let footer = options.footer ?? ""
let small = footer != ""
? `<p style="margin:24px 0 0;font-size:12px;line-height:1.5;color:#8a8a84">${escape(footer)}</p>`
: ""
let html = `<!doctype html><html lang="${escape(options.lang ?? "en")}"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>${escape(options.title ?? "")}</title></head><body style="margin:0;padding:0;background:#f6f6f3">${hidden}<table role="presentation" width="100%" style="border-collapse:collapse"><tr><td align="center" style="padding:24px 12px"><table role="presentation" width="100%" style="max-width:560px;border-collapse:collapse;background:#ffffff;border-radius:8px"><tr><td style="padding:28px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,Helvetica,Arial,sans-serif;font-size:15px;line-height:1.55;color:#1c1c1a">${body}${small}</td></tr></table></td></tr></table></body></html>`
let plain = content.text
return { html, text: footer != "" ? `${plain}\n\n--\n${footer}\n` : `${plain}\n` }
}

3
email/tests/Broken.weft Normal file
View file

@ -0,0 +1,3 @@
let missing: string? = null
---
<Text>{missing!}</Text>

7
email/tests/Escapes.weft Normal file
View file

@ -0,0 +1,7 @@
import A from "../components/A"
import P from "../components/P"
prop text: string
prop url: string
---
<P>{text}</P>
<P><A href={url}>go</A></P>

View file

@ -0,0 +1,21 @@
import A from "../components/A"
import Button from "../components/Button"
import Em from "../components/Em"
import Facts from "../components/Facts"
import H1 from "../components/H1"
import Hr from "../components/Hr"
import Img from "../components/Img"
import Ol from "../components/Ol"
import P from "../components/P"
import Strong from "../components/Strong"
import Ul from "../components/Ul"
---
<H1>Thank you</H1>
<P>Hi <Strong>Ana</Strong>, see <A href="https://x.dev/t">your ticket</A>.</P>
<Ul items={["One", "Two"]} />
<Ol items={["First", "Second"]} />
<Hr />
<Facts rows={[["Total", "19,99 €"], ["Date", "1. 12."]]} />
<Button url="https://x.dev/pay" label="Pay" />
<Img src="https://x.dev/a.png" alt="Logo" />
<P><Em>small</Em></P>

5
email/tests/Mailto.weft Normal file
View file

@ -0,0 +1,5 @@
import A from "../components/A"
import P from "../components/P"
---
<P><A href="mailto:a@b.co">write to us</A></P>
<P><A href="https://x.dev">https://x.dev</A></P>

8
email/tests/Unsafe.weft Normal file
View file

@ -0,0 +1,8 @@
import A from "../components/A"
import Button from "../components/Button"
import Img from "../components/Img"
import P from "../components/P"
---
<P><A href="javascript:alert(1)">click</A></P>
<Button url="javascript:alert(1)" label="press" />
<Img src="javascript:x" alt="a cat" />

451
email/tests/email.test.weft Normal file
View file

@ -0,0 +1,451 @@
// `weft test tests/email.test.weft`: messages, MIME, the providers' payloads and templates. packages/integration/email sends
// through both providers' APIs against a scripted server, and checks the MIME with Python's parser.
import { assert, assertEq, assertPanics, mount } from "std/testing"
import { stringify } from "std/json"
import { allowlist, clearOutbox, isValidEmail, memory, outbox, send, validate } from "../email"
import type { EmailError, Message } from "../email"
import { encodeHeader, formatAddress, quotedPrintable, toMime } from "../mime"
import * as postmark from "../postmark"
import * as resend from "../resend"
import { render } from "std/app"
import { document } from "../template"
import Broken from "./Broken"
import Escapes from "./Escapes"
import Everything from "./Everything"
import Mailto from "./Mailto"
import Unsafe from "./Unsafe"
fn message(): Message {
return { from: "info@example.com", to: ["ana@example.com"], subject: "Hello", text: "Hi" }
}
fn edit(change: (mut m: Message) => void): Message {
let mut m = message()
change(&m)
return m
}
fn problem(m: Message): string {
let _, err = try validate(m)
return match err {
InvalidMessage { field } => field,
null => "valid",
_ => "other",
}
}
// ---- validation ----
test "a good message is valid" {
assertEq(problem(message()), "valid")
assertEq(
problem(
edit((mut m) => {
m.to = [{ email: "ana@example.com", name: "Ana Novak" }]
m.html = "<p>Hi</p>"
}),
),
"valid",
)
}
test "addresses" {
for good of ["a@b.co", "first.last+tag@sub.example.com", "čebela@example.si"] {
assert(isValidEmail(good), good)
}
for bad of [
"",
"a",
"a@",
"@b.co",
"a@@b.co",
"a b@c.co",
"a@b..co",
"a@.b.co",
"a@b.co.",
"a<b@c.co",
"a,b@c.co",
"a@b.co\r\nBcc: x@y.z",
] {
assert(!isValidEmail(bad), bad)
}
}
test "what a message can't be sent without" {
assertEq(
problem(
edit((mut m) => {
m.to = []
}),
),
"to",
)
assertEq(
problem(
edit((mut m) => {
m.subject = " "
}),
),
"subject",
)
assertEq(
problem(
edit((mut m) => {
m.text = ""
}),
),
"html",
)
assertEq(
problem(
edit((mut m) => {
m.from = "nobody"
}),
),
"from",
)
assertEq(
problem(
edit((mut m) => {
m.to = ["nobody"]
}),
),
"to",
)
assertEq(
problem(
edit((mut m) => {
m.cc = ["nobody"]
}),
),
"cc",
)
assertEq(
problem(
edit((mut m) => {
m.bcc = ["nobody"]
}),
),
"bcc",
)
assertEq(
problem(
edit((mut m) => {
m.replyTo = "nobody"
}),
),
"replyTo",
)
}
test "header injection is refused" {
assertEq(
problem(
edit((mut m) => {
m.subject = "Hi\r\nBcc: victim@example.com"
}),
),
"subject",
)
assertEq(
problem(
edit((mut m) => {
m.subject = "Hi\nBcc: victim@example.com"
}),
),
"subject",
)
assertEq(
problem(
edit((mut m) => {
m.from = { email: "a@b.co", name: "A\r\nBcc: v@x.y" }
}),
),
"from",
)
assertEq(
problem(
edit((mut m) => {
m.to = [{ email: "a@b.co", name: "A\nX: y" }]
}),
),
"to",
)
assertEq(
problem(
edit((mut m) => {
m.headers = [["X-Tag", "a\r\nBcc: v@x.y"]]
}),
),
"headers",
)
assertEq(
problem(
edit((mut m) => {
m.headers = [["Bad Name", "a"]]
}),
),
"headers",
)
assertEq(
problem(
edit((mut m) => {
m.headers = [["X:Y", "a"]]
}),
),
"headers",
)
assertEq(
problem(
edit((mut m) => {
m.attachments = [{ filename: "a\r\nb.txt", content: Bytes.of([1]) }]
}),
),
"attachments",
)
}
// ---- sending ----
test "send validates, then hands the message to the transport" {
clearOutbox()
let sent = try await send(memory(), message())
assertEq(sent.provider, "memory")
assertEq(outbox().length, 1)
let _, err = try await send(
memory(),
edit((mut m) => {
m.to = []
}),
)
assert(err != null)
assertEq(outbox().length, 1)
}
test "an allowlist keeps other recipients from being sent to" {
clearOutbox()
let staging = allowlist(memory(), ["info@vidni.si"])
let skipped = try await send(staging, message())
assert(skipped.skipped)
assertEq(outbox().length, 0)
let sent = try await send(
staging,
edit((mut m) => {
m.to = ["INFO@vidni.si "]
}),
) else null
assert(sent == null, "an address with a space is invalid")
let allowed = try await send(
staging,
edit((mut m) => {
m.to = ["INFO@vidni.si"]
}),
)
assert(!allowed.skipped)
assertEq(outbox().length, 1)
// One recipient outside the list keeps the whole message back.
let mixed = try await send(
staging,
edit((mut m) => {
m.to = ["info@vidni.si"]
m.cc = ["ana@example.com"]
}),
)
assert(mixed.skipped)
let open = try await send(allowlist(memory(), ["*"]), message())
assert(!open.skipped)
}
// ---- MIME ----
test "headers stay ASCII" {
assertEq(encodeHeader("Hello"), "Hello")
assertEq(encodeHeader("Račun št. 5"), "=?UTF-8?B?UmHEjXVuIMWhdC4gNQ==?=")
// A long subject is split between characters, each encoded word at most 75 characters long.
for word of encodeHeader("č".repeat(100)).split("\r\n ") {
assert(word.length <= 75, word)
}
assertEq(formatAddress("a@b.co"), "a@b.co")
assertEq(
formatAddress({ email: "a@b.co", name: "Ana \"N\" Novak" }),
"\"Ana \\\"N\\\" Novak\" <a@b.co>",
)
assertEq(formatAddress({ email: "a@b.co", name: "Žiga" }), "=?UTF-8?B?xb1pZ2E=?= <a@b.co>")
}
test "quoted-printable" {
assertEq(quotedPrintable("a=b"), "a=3Db")
assertEq(quotedPrintable("č"), "=C4=8D")
assertEq(quotedPrintable("line one\nline two"), "line one\r\nline two")
assertEq(quotedPrintable("trailing space "), "trailing space=20")
for line of quotedPrintable("x".repeat(500)).split("\r\n") {
assert(line.length <= 76, line)
}
// A multi-byte character isn't split across a soft line break.
for line of quotedPrintable("č".repeat(100)).split("\r\n") {
assert(line.length <= 76 && !line.replace("=", "").endsWith("C4="), line)
}
}
test "a text message" {
let mime = toMime(message(), { id: "fixed", date: "Tue, 17 Nov 2026 10:00:00 +0000" })
assertEq(
mime,
"From: info@example.com\r\nTo: ana@example.com\r\nSubject: Hello\r\nDate: Tue, 17 Nov 2026 10:00:00 +0000\r\nMessage-ID: <fixed@example.com>\r\nMIME-Version: 1.0\r\nContent-Type: text/plain; charset=UTF-8\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nHi\r\n",
)
}
test "html and text, attachments and inline images nest in the right order" {
let mime = toMime(
edit((mut m) => {
m.html = "<p>Hi</p><img src=\"cid:logo\">"
m.attachments = [
{ filename: "racun.pdf", content: Bytes.of([1, 2, 3]), type: "application/pdf" },
{ filename: "logo.png", content: Bytes.of([4]), type: "image/png", contentId: "logo" },
]
}),
{ id: "x" },
)
assert(mime.includes("multipart/mixed"))
assert(mime.indexOf("multipart/mixed") < mime.indexOf("multipart/related"))
assert(mime.indexOf("multipart/related") < mime.indexOf("multipart/alternative"))
assert(mime.includes("Content-Disposition: attachment; filename=\"racun.pdf\""))
assert(mime.includes("Content-Disposition: inline; filename=\"logo.png\"\r\nContent-ID: <logo>"))
assert(mime.includes("\r\n\r\nAQID\r\n"))
}
test "bcc is never written, and every line ends in CRLF" {
let mime = toMime(
edit((mut m) => {
m.bcc = ["secret@example.com"]
m.html = "<p>a\nb</p>"
}),
{ id: "x" },
)
assert(!mime.includes("secret@example.com"))
assert(!mime.replaceAll("\r\n", "").includes("\n"))
}
test "a file name outside ASCII" {
let mime = toMime(
edit((mut m) => {
m.attachments = [{ filename: "račun.pdf", content: Bytes.of([1]) }]
}),
{ id: "x" },
)
assert(mime.includes("filename*=UTF-8''ra%C4%8Dun.pdf"))
}
// ---- the providers ----
fn full(): Message {
return {
from: { email: "info@example.com", name: "Vidni" },
to: ["ana@example.com", { email: "bor@example.com", name: "Bor" }],
cc: ["c@example.com"],
bcc: ["d@example.com"],
replyTo: "owner@example.com",
subject: "Račun",
html: "<p>Hi</p>",
text: "Hi",
headers: [["List-Unsubscribe", "<https://example.com/u>"]],
attachments: [
{
filename: "a.ics",
content: Bytes.fromUtf8("BEGIN"),
type: "text/calendar",
contentId: "cal",
},
],
}
}
test "Resend's payload" {
assertEq(
stringify(resend.payload(full())),
"{\"from\":\"\\\"Vidni\\\" <info@example.com>\",\"to\":[\"ana@example.com\",\"\\\"Bor\\\" <bor@example.com>\"],\"subject\":\"Račun\",\"cc\":[\"c@example.com\"],\"bcc\":[\"d@example.com\"],\"reply_to\":\"owner@example.com\",\"html\":\"<p>Hi</p>\",\"text\":\"Hi\",\"headers\":{\"List-Unsubscribe\":\"<https://example.com/u>\"},\"attachments\":[{\"filename\":\"a.ics\",\"content\":\"QkVHSU4=\",\"content_type\":\"text/calendar\",\"content_id\":\"cal\"}]}",
)
assertEq(
stringify(resend.payload(message())),
"{\"from\":\"info@example.com\",\"to\":[\"ana@example.com\"],\"subject\":\"Hello\",\"text\":\"Hi\"}",
)
}
test "Postmark's payload" {
assertEq(
stringify(postmark.payload(full(), "broadcast")),
"{\"From\":\"\\\"Vidni\\\" <info@example.com>\",\"To\":\"ana@example.com, \\\"Bor\\\" <bor@example.com>\",\"Subject\":\"Račun\",\"MessageStream\":\"broadcast\",\"Cc\":\"c@example.com\",\"Bcc\":\"d@example.com\",\"ReplyTo\":\"owner@example.com\",\"HtmlBody\":\"<p>Hi</p>\",\"TextBody\":\"Hi\",\"Headers\":[{\"Name\":\"List-Unsubscribe\",\"Value\":\"<https://example.com/u>\"}],\"Attachments\":[{\"Name\":\"a.ics\",\"Content\":\"QkVHSU4=\",\"ContentType\":\"text/calendar\",\"ContentID\":\"cid:cal\"}]}",
)
}
fn kind(e: EmailError): string {
return e.kind
}
test "providers' errors" {
assertEq(kind(resend.failureOf(422, null, null)), "Rejected")
assertEq(kind(resend.failureOf(401, null, null)), "Rejected")
assertEq(kind(resend.failureOf(500, null, null)), "ProviderFailure")
assertEq(
resend.failureOf(429, null, "30"),
RateLimited { message: "Resend answered 429", retryAfter: 30 },
)
assertEq(kind(postmark.failureOf(422, null)), "Rejected")
assertEq(kind(postmark.failureOf(429, null)), "RateLimited")
assertEq(kind(postmark.failureOf(503, null)), "ProviderFailure")
}
// ---- templates ----
test "everything shown in a template is escaped" {
let r = render(Escapes, { text: "<script>alert(1)</script> & co", url: "https://x.dev/?a=1&b=2" })
assert(!r.html.includes("<script>"))
assert(r.html.includes("&lt;script&gt;alert(1)&lt;/script&gt; &amp; co"))
assert(r.html.includes("href=\"https://x.dev/?a=1&amp;b=2\""))
}
test "unsafe links and images lose their address" {
let r = render(Unsafe)
assert(!r.html.includes("javascript:"))
assert(r.html.includes("click") && r.html.includes("press") && r.html.includes("a cat"))
assert(!r.text.includes("javascript:"))
}
test "the text version reads like a person wrote it" {
assertEq(
render(Everything).text,
"Thank you\n\nHi Ana, see your ticket (https://x.dev/t).\n\n- One\n- Two\n\n1. First\n2. Second\n\n---\n\nTotal: 19,99 €\nDate: 1. 12.\n\nPay (https://x.dev/pay)\n\nLogo\n\nsmall",
)
}
test "a mailto link's text doesn't repeat the address" {
assertEq(render(Mailto).text, "write to us (a@b.co)\n\nhttps://x.dev")
}
test "styles are inline, in forms mail clients read" {
let html = render(Everything).html
assert(html.includes("font-weight:700"))
assert(html.includes("margin-bottom:16px"))
assert(html.includes("#111111"))
assert(html.includes("alt=\"Logo\""))
for missing of ["class=", "var(", "oklch(", "rem", "<!--"] {
assert(!html.includes(missing), missing)
}
}
test "rendering leaves what is mounted alone" {
let view = mount(Mailto)
let before = view.text()
render(Everything)
assertEq(view.text(), before)
}
test "a panic while rendering is the caller's" {
assertPanics(() => render(Broken), "TypeError")
}
test "a whole document" {
let r = document(
render(Mailto),
{ title: "Receipt <1>", preheader: "Thanks", footer: "Vidni d.o.o.", lang: "sl" },
)
assert(r.html.startsWith("<!doctype html><html lang=\"sl\">"))
assert(r.html.includes("<title>Receipt &lt;1&gt;</title>"))
assert(r.html.includes("Thanks") && r.html.includes("Vidni d.o.o."))
assertEq(r.text, "write to us (a@b.co)\n\nhttps://x.dev\n\n--\nVidni d.o.o.\n")
}

11
email/weft.toml Normal file
View file

@ -0,0 +1,11 @@
[package]
name = "weft/email"
version = "0.1.0"
description = "Email for Weft: Resend and Postmark, MIME messages with attachments, and templates rendered to HTML and text"
license = "MIT OR Apache-2.0"
# A placeholder until the public repository exists (packages/README.md, Publishing for real).
repository = "https://git.weft-lang.dev/weft/packages"
hosts = ["api.resend.com", "api.postmarkapp.com"]
exports = ["email", "mime", "resend", "postmark", "template", "components"]
main = "email"
include = ["*.weft", "components", "README.md"]

1
integration/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
weft.lock

232
integration/email/main.weft Normal file
View file

@ -0,0 +1,232 @@
// `node packages/email/scripts/integration.mjs` runs this: both providers' APIs against a scripted server, and a full message
// printed for Python's `email` parser to read back.
import { header, json as jsonResponse, serve, status } from "std/http"
import type { Request } from "std/http"
import { parse } from "std/json"
import { log } from "std/log"
import { exit, stdout } from "std/process"
import { send } from "@weft/email"
import type { EmailError, Message } from "@weft/email"
import { toMime } from "@weft/email/mime"
import { postmark } from "@weft/email/postmark"
import { resend } from "@weft/email/resend"
let mut passed = 0
let mut failed = 0
fn check(name: string, ok: boolean, detail: string = "") {
if ok {
passed += 1
} else {
failed += 1
log(`FAIL ${name} ${detail}`)
}
}
let mut seen: Request[] = []
let mut script: string[] = []
fn okResend(): json {
return { id: "re_123" }
}
fn okPostmark(): json {
return {
To: "ana@example.com",
SubmittedAt: "2026-11-17T10:00:00Z",
MessageID: "pm-123",
ErrorCode: 0,
Message: "OK",
}
}
fn refused(): json {
return { statusCode: 422, name: "validation_error", message: "The domain is not verified." }
}
fn postmarkRefused(): json {
return { ErrorCode: 406, Message: "The address is inactive." }
}
async fn handle(req: Request) {
seen.push(req)
let step = script.shift() ?? "ok"
return match step {
"resend-ok" => jsonResponse(okResend()),
"postmark-ok" => jsonResponse(okPostmark()),
"resend-422" => jsonResponse(refused(), { status: 422 }),
"postmark-422" => jsonResponse(postmarkRefused(), { status: 422 }),
"postmark-200-error" => jsonResponse(postmarkRefused()),
"429" => header(jsonResponse(refused(), { status: 429 }), "retry-after", "12"),
"500" => status(500, "broken"),
"no-id" => jsonResponse(refused()),
_ => status(404),
}
}
fn kindOf(e: EmailError?): string {
return e == null ? "none" : e.kind
}
fn message(): Message {
return {
from: { email: "info@example.com", name: "Vidni" },
to: ["ana@example.com"],
subject: "Vaša vstopnica",
html: "<p>Živjo</p>",
text: "Živjo",
attachments: [
{ filename: "vstopnica.pdf", content: Bytes.of([37, 80, 68, 70]), type: "application/pdf" },
],
}
}
fn edit(change: (mut m: Message) => void): Message {
let mut m = message()
change(&m)
return m
}
fn parsed(req: Request): json {
return try parse(req.text()) else null
}
async fn main() {
let server = try await serve({ port: 0, host: "127.0.0.1" }, handle)
let base = `http://127.0.0.1:${server.port}`
// ---- Resend ----
let viaResend = resend(
{ apiKey: "re_key", baseUrl: base, idempotencyKey: m => `order-${m.subject.length}` },
)
script = ["resend-ok"]
seen = []
let sent1, e0 = try await send(viaResend, message())
check("Resend: sent", sent1?.id == "re_123" && sent1?.provider == "resend", kindOf(e0))
check("Resend: the endpoint", seen[0].path == "/emails" && seen[0].method == "POST", seen[0].path)
check("Resend: the key", seen[0].headers.get("authorization") == "Bearer re_key")
check("Resend: the idempotency key", seen[0].headers.get("idempotency-key") == "order-14")
check("Resend: JSON", (seen[0].headers.get("content-type") ?? "").startsWith("application/json"))
check(
"Resend: the fields",
parsed(seen[0]).subject == "Vaša vstopnica" &&
parsed(seen[0]).attachments[0].content == "JVBERg==",
)
script = ["resend-422"]
let _, e1 = try await send(viaResend, message())
match e1 {
Rejected { status: s, code, message: m } => check(
"Resend: a refusal",
s == 422 && code == "validation_error" && m == "The domain is not verified.",
),
_ => check("Resend: a refusal", false, kindOf(e1)),
}
script = ["429"]
let _, e2 = try await send(viaResend, message())
match e2 {
RateLimited { retryAfter } => check("Resend: rate limited", retryAfter == 12),
_ => check("Resend: rate limited", false, kindOf(e2)),
}
script = ["500"]
let _, e3 = try await send(viaResend, message())
check("Resend: a server error", kindOf(e3) == "ProviderFailure", kindOf(e3))
script = ["no-id"]
let _, e4 = try await send(viaResend, message())
check("Resend: an answer without an id", kindOf(e4) == "BadResponse", kindOf(e4))
// An invalid message never reaches the provider.
seen = []
let _, e5 = try await send(
viaResend,
edit((mut m) => {
m.subject = "a\r\nBcc: v@x.y"
}),
)
check(
"an invalid message isn't sent",
kindOf(e5) == "InvalidMessage" && seen.length == 0,
kindOf(e5),
)
// ---- Postmark ----
let viaPostmark = postmark({ serverToken: "pm_token", baseUrl: base })
script = ["postmark-ok"]
seen = []
let sent2, e10 = try await send(viaPostmark, message())
check("Postmark: sent", sent2?.id == "pm-123" && sent2?.provider == "postmark", kindOf(e10))
check(
"Postmark: the endpoint",
seen[0].path == "/email" && seen[0].method == "POST",
seen[0].path,
)
check("Postmark: the token", seen[0].headers.get("x-postmark-server-token") == "pm_token")
check(
"Postmark: the fields",
parsed(seen[0]).Subject == "Vaša vstopnica" &&
parsed(seen[0]).MessageStream == "outbound" &&
parsed(seen[0]).Attachments[0].Name == "vstopnica.pdf",
)
script = ["postmark-422"]
let _, e6 = try await send(viaPostmark, message())
match e6 {
Rejected { code, message: m } => check(
"Postmark: a refusal",
code == "406" && m == "The address is inactive.",
),
_ => check("Postmark: a refusal", false, kindOf(e6)),
}
script = ["postmark-200-error"]
let _, e7 = try await send(viaPostmark, message())
check("Postmark: an error code in a 200", kindOf(e7) == "Rejected", kindOf(e7))
script = ["500"]
let _, e8 = try await send(viaPostmark, message())
check("Postmark: a server error", kindOf(e8) == "ProviderFailure", kindOf(e8))
// ---- nothing listening ----
let dead = resend({ apiKey: "k", baseUrl: "http://127.0.0.1:9", timeout: 2000 })
let _, e9 = try await send(dead, message())
check("no connection", kindOf(e9) == "ConnectionFailed", kindOf(e9))
await server.close()
// ---- a full message for Python's parser ----
let long = "Pozdravljeni, to je dolgo sporočilo s črkami č, š in ž, ki sega čez mejo šestinsedemdesetih znakov v vrstici. = in pika.\n.vrstica s piko\nKonec "
let full: Message = {
from: { email: "info@example.com", name: "Žiga \"Vidni\" Novak" },
to: ["ana@example.com", { email: "bor@example.com", name: "Bor Šuštar" }],
cc: ["cene@example.com"],
bcc: ["skrito@example.com"],
replyTo: { email: "lastnik@example.com", name: "Lastnik" },
subject: "Vaša vstopnica za dogodek: čudovit večer na gradu, ki se ga boste še dolgo spominjali",
text: long,
html: `<p>${long}</p><img src="cid:logo">`,
headers: [["List-Unsubscribe", "<https://example.com/odjava?id=1>"], ["X-Oznaka", "račun"]],
attachments: [
{
filename: "račun št. 5.pdf",
content: Bytes.of((0..=255).toList()),
type: "application/pdf",
},
{
filename: "dogodek.ics",
content: Bytes.fromUtf8("BEGIN:VCALENDAR\r\nEND:VCALENDAR\r\n"),
type: "text/calendar",
},
{
filename: "logo.png",
content: Bytes.of([137, 80, 78, 71]),
type: "image/png",
contentId: "logo",
},
],
}
stdout("=====MIME=====\n" + toMime(full) + "=====END=====\n")
log(`${passed} passed, ${failed} failed`)
if failed > 0 {
exit(1)
}
}
try await main()

View file

@ -0,0 +1,5 @@
# The integration tests of `weft/email`, an app that depends on the package: `node packages/email/scripts/integration.mjs`.
main = "main.weft"
[dependencies]
"weft/email" = { path = "../../email", allow = ["http.fetch@api.resend.com", "http.fetch@api.postmarkapp.com", "http.fetch@127.0.0.1", "http.media"] }

View file

@ -0,0 +1,499 @@
// `node packages/stripe/scripts/integration.mjs` runs this: every call against stripe-mock (Stripe's OpenAPI spec checks each request, and
// its fixtures check each response type), then retries, headers and paging against a small server in this program.
import { header, json as jsonResponse, serve, status, text } from "std/http"
import type { Request } from "std/http"
import { log } from "std/log"
import { env, exit } from "std/process"
import { client, errorMessage, idOf, request } from "@weft/stripe"
import type { Client, StripeError } from "@weft/stripe"
import * as billing from "@weft/stripe/billing"
import * as checkout from "@weft/stripe/checkout"
import * as customers from "@weft/stripe/customers"
import * as paymentIntents from "@weft/stripe/payment_intents"
import * as prices from "@weft/stripe/prices"
import * as products from "@weft/stripe/products"
import * as refunds from "@weft/stripe/refunds"
import * as subscriptions from "@weft/stripe/subscriptions"
let mut passed = 0
let mut failed = 0
fn check(name: string, ok: boolean, detail: string = "") {
if ok {
passed += 1
} else {
failed += 1
log(`FAIL ${name} ${detail}`)
}
}
fn failure(name: string, e: StripeError) {
failed += 1
log(`FAIL ${name}: ${errorMessage(e)}`)
}
let mock = client(
"sk_test_123",
{ baseUrl: env("STRIPE_MOCK_URL") ?? "http://127.0.0.1:12111", maxRetries: 0 },
)
// ---- every call, against Stripe's OpenAPI spec ----
async fn runMock() {
let customer, e1 = try await customers.create(
mock,
{
email: "ana@example.com",
name: "Ana",
address: { city: "Ljubljana", country: "SI" },
metadata: Map<string, string>([["site", "a1"]]),
preferred_locales: ["sl"],
},
)
if e1 != null {
return failure("customers.create", e1)
}
check("customers.create", customer.id != "")
let got, e2 = try await customers.retrieve(mock, customer.id)
if e2 != null {
return failure("customers.retrieve", e2)
}
check("customers.retrieve", got.id != "")
let up, e3 = try await customers.update(
mock,
customer.id,
{ name: "Ana N." },
{ idempotencyKey: "k1" },
)
check("customers.update", e3 == null)
let page, e4 = try await customers.list(mock, { limit: 3, email: "ana@example.com" })
check("customers.list", e4 == null)
let all, e5 = try await customers.listAll(mock, { limit: 3 })
check("customers.listAll", e5 == null)
let found, e6 = try await customers.search(mock, "email:\"ana@example.com\"")
check("customers.search", e6 == null)
let gone, e7 = try await customers.remove(mock, customer.id)
check("customers.remove", e7 == null && gone.deleted)
let product, e8 = try await products.create(
mock,
{ name: "Ticket", active: true, metadata: Map<string, string>([["a", "b"]]) },
)
if e8 != null {
return failure("products.create", e8)
}
check("products.create", product.id != "")
check("products.retrieve", (try await products.retrieve(mock, product.id) else null) != null)
check(
"products.update",
(try await products.update(mock, product.id, { description: "x" }) else null) != null,
)
check(
"products.list",
(try await products.list(mock, { active: true, limit: 2 }) else null) != null,
)
check("products.listAll", (try await products.listAll(mock) else null) != null)
check("products.remove", (try await products.remove(mock, product.id) else null) != null)
let price, e9 = try await prices.create(
mock,
{
currency: "eur",
unit_amount: 1999,
product: product.id,
recurring: { interval: "month", interval_count: 1 },
lookup_key: "base-monthly",
},
)
if e9 != null {
return failure("prices.create", e9)
}
check("prices.create", price.id != "")
check("prices.retrieve", (try await prices.retrieve(mock, price.id) else null) != null)
check(
"prices.update",
(try await prices.update(mock, price.id, { nickname: "Base", active: true }) else null) != null,
)
let keyed, e10 = try await prices.list(
mock,
{ lookup_keys: ["base-monthly"], active: true, limit: 1 },
)
check("prices.list by lookup key", e10 == null)
check(
"prices.listAll",
(try await prices.listAll(mock, { product: product.id }) else null) != null,
)
let intent, e11 = try await paymentIntents.create(
mock,
{
amount: 1999,
currency: "eur",
automatic_payment_methods: { enabled: true },
metadata: Map<string, string>([["sale", "s1"]]),
application_fee_amount: 100,
transfer_data: { destination: "acct_1" },
},
{ idempotencyKey: "sale-s1", stripeAccount: "acct_1" },
)
if e11 != null {
return failure("paymentIntents.create", e11)
}
check("paymentIntents.create", intent.id != "" && intent.amount > 0)
check(
"paymentIntents.retrieve",
(try await paymentIntents.retrieve(mock, intent.id) else null) != null,
)
check(
"paymentIntents.update",
(try await paymentIntents.update(mock, intent.id, { description: "x" }) else null) != null,
)
check(
"paymentIntents.confirm",
(try await paymentIntents.confirm(mock, intent.id) else null) != null,
)
check(
"paymentIntents.capture",
(try await paymentIntents.capture(mock, intent.id) else null) != null,
)
check(
"paymentIntents.cancel",
(try await paymentIntents.cancel(mock, intent.id) else null) != null,
)
check(
"paymentIntents.list",
(try await paymentIntents.list(mock, { limit: 2 }) else null) != null,
)
check("paymentIntents.listAll", (try await paymentIntents.listAll(mock) else null) != null)
check(
"paymentIntents.search",
(try await paymentIntents.search(mock, "status:\"succeeded\"") else null) != null,
)
let sub, e12 = try await subscriptions.create(
mock,
{
customer: "cus_1",
items: [{ price: price.id, quantity: 2 }],
metadata: Map<string, string>([["siteId", "a1"]]),
trial_period_days: 14,
},
)
if e12 != null {
return failure("subscriptions.create", e12)
}
check("subscriptions.create", sub.id != "" && idOf(sub.customer) != null)
let item = sub.items.data.at(0)
check("a subscription's items carry their price", item != null && item.price != null)
check(
"subscriptions.retrieve",
(try await subscriptions.retrieve(mock, sub.id, { expand: ["customer"] }) else null) != null,
)
check(
"subscriptions.update",
(try await subscriptions.update(
mock,
sub.id,
{
cancel_at_period_end: true,
items: [{ id: "si_1", deleted: true }, { price: price.id, quantity: 1 }],
proration_behavior: "none",
},
) else null) !=
null,
)
check("subscriptions.cancel", (try await subscriptions.cancel(mock, sub.id) else null) != null)
check(
"subscriptions.list",
(try await subscriptions.list(mock, { status: "all", limit: 2 }) else null) != null,
)
check(
"subscriptions.listAll",
(try await subscriptions.listAll(mock, { customer: "cus_1" }) else null) != null,
)
check(
"subscriptions.search",
(try await subscriptions.search(mock, "status:\"active\"") else null) != null,
)
let session, e13 = try await checkout.create(
mock,
{
mode: "payment",
success_url: "https://example.com/thanks?s={CHECKOUT_SESSION_ID}",
cancel_url: "https://example.com/cart",
customer_email: "ana@example.com",
client_reference_id: "sale-1",
locale: "sl",
metadata: Map<string, string>([["saleId", "s1"]]),
line_items: [
{
quantity: 1,
price_data: { currency: "eur", unit_amount: 1999, product_data: { name: "Vstopnica" } },
},
],
payment_intent_data: {
application_fee_amount: 100,
description: "Ticket",
metadata: Map<string, string>([["saleId", "s1"]]),
},
},
{ idempotencyKey: "vidni-sale-s1", stripeAccount: "acct_1" },
)
if e13 != null {
return failure("checkout.create", e13)
}
check("checkout.create", session.id != "")
let sub2, e14 = try await checkout.create(
mock,
{
mode: "subscription",
success_url: "https://example.com/ok",
line_items: [{ price: price.id, quantity: 1 }],
allow_promotion_codes: true,
subscription_data: {
metadata: Map<string, string>([["siteId", "a1"]]),
trial_period_days: 7,
},
},
)
check("checkout.create for a subscription", e14 == null)
check(
"checkout.retrieve",
(try await checkout.retrieve(
mock,
session.id,
{ expand: ["subscription", "payment_intent"] },
) else null) !=
null,
)
check("checkout.expire", (try await checkout.expire(mock, session.id) else null) != null)
check("checkout.list", (try await checkout.list(mock, { limit: 2 }) else null) != null)
check("checkout.listAll", (try await checkout.listAll(mock) else null) != null)
check("checkout.lineItems", (try await checkout.lineItems(mock, session.id) else null) != null)
let refund, e15 = try await refunds.create(
mock,
{ payment_intent: intent.id, amount: 100, reason: "requested_by_customer" },
)
if e15 != null {
return failure("refunds.create", e15)
}
check("refunds.create", refund.id != "")
check("refunds.retrieve", (try await refunds.retrieve(mock, refund.id) else null) != null)
check(
"refunds.list",
(try await refunds.list(mock, { payment_intent: intent.id }) else null) != null,
)
check("refunds.listAll", (try await refunds.listAll(mock) else null) != null)
// That stripe-mock really checks requests: a field Stripe doesn't have is refused.
let unknown: json = { not_a_field: "x" }
let _, e18 = try await customers.create(mock, { extra: unknown })
check(
"stripe-mock refuses a field Stripe doesn't have",
kindOf(e18) == "InvalidRequest",
kindOf(e18),
)
let portal, e16 = try await billing.createPortalSession(
mock,
{ customer: "cus_1", return_url: "https://example.com" },
)
check("billing.createPortalSession", e16 == null)
check(
"billing.retrieveCoupon",
(try await billing.retrieveCoupon(mock, "co_1") else null) != null,
)
check(
"billing.retrievePromotionCode",
(try await billing.retrievePromotionCode(mock, "promo_1") else null) != null,
)
let components: json = { payments: { enabled: true } }
let account, e17 = try await billing.createAccountSession(mock, { account: "acct_1", components })
check("billing.createAccountSession", e17 == null)
}
// ---- what stripe-mock can't show: a server that misbehaves on request ----
let mut seen: Request[] = []
let mut script: string[] = []
async fn handle(req: Request) {
seen.push(req)
let step = script.shift() ?? "ok"
return match step {
"500" => status(500, "{\"error\":{\"type\":\"api_error\",\"message\":\"boom\"}}"),
"500-no-retry" => header(
status(500, "{\"error\":{\"type\":\"api_error\",\"message\":\"no retry\"}}"),
"stripe-should-retry",
"false",
),
"429" => jsonResponse(rateLimit(), { status: 429 }),
"402" => header(jsonResponse(declined(), { status: 402 }), "request-id", "req_declined"),
"garbage" => text("this is not json"),
"wrong-shape" => jsonResponse(wrongShape()),
"page1" => jsonResponse(pageOf(["cus_1", "cus_2"], true)),
"page2" => jsonResponse(pageOf(["cus_3"], false)),
_ => jsonResponse(okCustomer()),
}
}
fn rateLimit(): json {
return { error: { type: "invalid_request_error", message: "slow down" } }
}
fn declined(): json {
return {
error: {
type: "card_error",
code: "card_declined",
decline_code: "insufficient_funds",
message: "Your card was declined.",
},
}
}
fn wrongShape(): json {
return { id: 42 }
}
fn okCustomer(): json {
return { id: "cus_ok", object: "customer" }
}
fn pageOf(ids: string[], more: boolean): json {
let data: json[] = ids.map(id => {
let item: json = { id }
return item
})
return { object: "list", data, has_more: more }
}
fn kindOf(e: StripeError?): string {
return e == null ? "none" : e.kind
}
async fn runScripted(c: Client) {
// A server error is repeated, with the same idempotency key, and then succeeds.
script = ["500", "ok"]
seen = []
let first, e1 = try await customers.create(c, { email: "a@b.c" })
check("a 500 is retried", e1 == null && seen.length == 2, kindOf(e1))
let keys = seen.map(r => r.headers.get("idempotency-key") ?? "")
check("every attempt carries one idempotency key", keys[0] != "" && keys[0] == keys[1])
// An explicit key is what is sent.
script = ["ok"]
seen = []
let _, e2 = try await customers.create(
c,
{ email: "a@b.c" },
{ idempotencyKey: "my-key", stripeAccount: "acct_9" },
)
check(
"idempotency key and connected account",
e2 == null &&
seen[0].headers.get("idempotency-key") == "my-key" &&
seen[0].headers.get("stripe-account") == "acct_9",
)
check(
"the API key and version are sent",
seen[0].headers.get("authorization") == "Bearer sk_test_scripted" &&
(seen[0].headers.get("stripe-version") ?? "") != "",
)
check(
"a POST is form encoded",
seen[0].headers.get("content-type") == "application/x-www-form-urlencoded" &&
seen[0].text() == "email=a%40b.c",
)
// Reads have no key, and carry their parameters in the query.
script = ["ok"]
seen = []
let _, e3 = try await customers.retrieve(c, "cus/1", { expand: ["default_source"] })
check(
"a GET has no idempotency key",
e3 == null && seen[0].headers.get("idempotency-key") == null,
)
check(
"expand goes in the query, the id is escaped",
seen[0].url.includes("/v1/customers/cus%2F1?expand[0]=default_source"),
seen[0].url,
)
// Stripe saying not to retry is obeyed.
script = ["500-no-retry", "ok"]
seen = []
let _, e4 = try await customers.create(c, { email: "a@b.c" })
check("stripe-should-retry: false", kindOf(e4) == "ApiFailure" && seen.length == 1, kindOf(e4))
// Retries run out.
script = ["500", "500", "500", "ok"]
seen = []
let _, e5 = try await customers.create(c, { email: "a@b.c" })
check("retries are limited", kindOf(e5) == "ApiFailure" && seen.length == 3, `${seen.length}`)
// A 429 is an error the caller decides about, unless Stripe asks for a retry.
script = ["429", "ok"]
seen = []
let _, e6 = try await customers.create(c, { email: "a@b.c" })
check("a 429 is a typed error", kindOf(e6) == "RateLimited" && seen.length == 1, kindOf(e6))
// A declined card.
script = ["402"]
seen = []
let _, e7 = try await customers.create(c, { email: "a@b.c" })
match e7 {
CardError { failure: f } => check(
"a declined card",
f.decline_code == "insufficient_funds" && f.request_id == "req_declined",
),
_ => check("a declined card", false, kindOf(e7)),
}
// What can't be read is an error, not a crash.
script = ["wrong-shape"]
let _, e8 = try await customers.retrieve(c, "cus_1")
check("an unexpected shape", kindOf(e8) == "BadResponse", kindOf(e8))
script = ["garbage"]
let _, e9 = try await customers.retrieve(c, "cus_1")
check("an answer that isn't JSON", kindOf(e9) == "BadResponse", kindOf(e9))
// Every page is followed.
script = ["page1", "page2"]
seen = []
let all, e10 = try await customers.listAll(c, { limit: 2 })
check(
"listAll follows the pages",
e10 == null && all.map(x => x.id).join(",") == "cus_1,cus_2,cus_3",
kindOf(e10),
)
check(
"listAll asks for the page after the last item",
seen.length == 2 && seen[1].url.includes("starting_after=cus_2"),
)
// And stops at the limit it was given.
script = ["page1", "page2"]
seen = []
let some, e11 = try await customers.listAll(c, {}, {}, 2)
check("listAll stops at max", e11 == null && some.length == 2 && seen.length == 1)
// Nothing listening.
let dead = client(
"sk_test_scripted",
{ baseUrl: "http://127.0.0.1:9", maxRetries: 1, timeout: 2000 },
)
let _, e12 = try await customers.retrieve(dead, "cus_1")
check("no connection", kindOf(e12) == "ConnectionFailed", kindOf(e12))
}
await runMock()
let server = try await serve({ port: 0, host: "127.0.0.1" }, handle)
await runScripted(
client("sk_test_scripted", { baseUrl: `http://127.0.0.1:${server.port}`, maxRetries: 2 }),
)
await server.close()
log(`${passed} passed, ${failed} failed`)
if failed > 0 {
exit(1)
}

View file

@ -0,0 +1,5 @@
# The integration tests of `weft/stripe`, an app that depends on the package: `node packages/stripe/scripts/integration.mjs`.
main = "main.weft"
[dependencies]
"weft/stripe" = { path = "../../stripe", allow = ["http.fetch@api.stripe.com", "http.fetch@127.0.0.1"] }

102
markdown/README.md Normal file
View file

@ -0,0 +1,102 @@
# @weft/markdown
CommonMark and GitHub Flavored Markdown for Weft: a syntax tree, HTML, and a sanitiser. It is written in Weft, needs no
capabilities (it does no I/O), and is safe for untrusted input by default.
```sh
weft add weft/markdown
```
```weft
import { parse, render, sanitize, toHtml, toText, visit } from "@weft/markdown"
toHtml("# Hello\n\n**bold** <script>alert(1)</script>")
// <h1>Hello</h1>
// <p><strong>bold</strong> &lt;script&gt;alert(1)&lt;/script&gt;</p>
```
## What it gives you
| | |
|---|---|
| `toHtml(source, options?)` | Markdown to HTML |
| `parse(source, options?)` | the syntax tree (`Document`, typed in `@weft/markdown/ast`) |
| `render(doc, options?)` | a tree to HTML, for a tree you have changed |
| `toText(doc)` | plain text, for previews, search and the text part of an email |
| `visit(doc, f)` | every inline of a document, to collect links or images |
| `sanitize(html, policy?)` | the allowlist sanitiser on its own, for HTML from anywhere |
GFM is on by default: tables, task lists, strikethrough and bare links (`www.`, `https://`, email addresses). `gfm: false`
gives plain CommonMark.
## Safe by default
- **Raw HTML is shown as text.** `html: "sanitize"` keeps what an allowlist permits; `html: "trust"` passes it through and
is for Markdown you wrote yourself.
- **Links and images only use `http`, `https`, `mailto` and `tel`**, or are relative (`schemes` changes the list). Images
may also be `data:image/` PNG, GIF, JPEG or WebP, never SVG. A link that isn't allowed renders as its text.
- **The sanitiser is an allowlist over a tokenizer**, and what it writes is well formed with every text escaped. Scripts,
styles, frames, forms, SVG, MathML, event handlers and unsafe URLs are removed. The only `<input>` kept is a disabled
checkbox.
- **Hostile documents can't take the server down.** Quotes, lists and inlines stop nesting at 64 levels, the text a link
reference expands to is bounded, and every input in `tests/hostile.weft` takes time proportional to its size.
```weft
toHtml("a <kbd>b</kbd> <script>c()</script>", { html: "sanitize" }) // <p>a <kbd>b</kbd> </p>
sanitize("<a href=\"javascript:x()\" onclick=\"y()\">hi</a>") // <a>hi</a>
```
## Options
```weft
toHtml(source, {
headingOffset: 1, // `# Title` renders as <h2>
linkTarget: "_blank", // adds rel="noopener noreferrer"
rewriteUrl: (url, kind) => url.startsWith("upload:") ? cdn(url.slice(7)) : null,
})
```
`rewriteUrl` runs before the safety check, so an app's own scheme (`upload:<id>`) can resolve to a real address, and one it
can't resolve is dropped rather than rendered.
## Conformance
`weft run tests/spec.weft` runs the specs' examples:
| | |
|---|---|
| CommonMark 0.31.2 | 652 of 652 |
| GitHub Flavored Markdown | 660 of 669, and the other 9 are below |
The 9 are examples where the two specs give different HTML for the same text: the GFM spec was written against CommonMark
0.29, and its renderer folds nested `<strong>`, which CommonMark 0.31.2's own examples keep. This package follows CommonMark
there, and the runner checks each such example against CommonMark's expected output. GFM's `tagfilter` example isn't run:
`html: "escape"` and `"sanitize"` already remove those tags, and `"trust"` means what it says.
## Speed
Against markdown-it 15.0.3 on Node 24, rendering the CommonMark spec's examples joined and repeated (`scripts/bench.mjs`;
one cold run each, in a 6 GB container, so markdown-it's JIT is warming up over the small sizes):
| document | Weft VM | markdown-it (JIT) | markdown-it (`--jitless`) |
|---|---|---|---|
| 16 KB | 52 ms | 44 ms | 47 ms |
| 157 KB | 112 ms | 105 ms | 242 ms |
| 787 KB | 1.2 s | 0.54 s | 1.6 s |
| 3.1 MB | 5.3 s | 1.5 s | 5.7 s |
## Limits
- A turn's default limits (32 MB of allocation, and 250 ms when serving) fit documents up to about 150 KB. Above that, raise
`WEFT_MEMORY_MB` and `WEFT_DEADLINE_MS`, or render outside a request.
## Development
```sh
weft test tests/markdown.test.weft # the API and its safety
weft run tests/spec.weft # the specs' examples (-v prints each failure)
node scripts/hostile.mjs # hostile input: nothing stops, nothing is quadratic
node scripts/bench.mjs commonmark.json # beside markdown-it (MARKDOWN_IT=<its folder>)
node scripts/gen-spec.mjs # regenerates tests/cases.weft from the specs
node scripts/gen-tables.mjs # regenerates lib/entities.weft and lib/unicode.weft
```

83
markdown/ast.weft Normal file
View file

@ -0,0 +1,83 @@
// The Markdown syntax tree and the options of every function in `@weft/markdown`.
/// A parsed document: its blocks, in order.
export type Document = { children: Block[] }
/// A table column's alignment, `null` when its delimiter row doesn't say.
export type Align = "left" | "center" | "right" | null
/// A block: what a document, a quote or a list item is made of.
///
/// - `CodeBlock`: `info` is the text after the opening fence (`""` for an indented block).
/// - `HtmlBlock`: raw HTML as written, rendered according to `Options.html`.
/// - `BulletList`, `OrderedList`: a `tight` list has no blank lines between its items; `start` is the first number.
/// - `Table`: a GFM table, with one cell per column in every row.
export type Block =
| Paragraph { children: Inline[] }
| Heading { level: float, children: Inline[] }
| ThematicBreak
| CodeBlock { info: string, code: string }
| HtmlBlock { html: string }
| Blockquote { children: Block[] }
| BulletList { tight: boolean, items: ListItem[] }
| OrderedList { tight: boolean, start: float, items: ListItem[] }
| Table { align: Align[], head: TableCell[], rows: TableCell[][] }
/// A list item. `checked` is `null` unless the item is a GFM task (`- [x] done`).
export type ListItem = { children: Block[], checked: boolean? }
/// A table cell.
export type TableCell = { children: Inline[] }
/// An inline: what paragraphs, headings and cells are made of. `HtmlInline` is raw HTML as written, rendered according
/// to `Options.html`.
export type Inline =
| Str { text: string }
| SoftBreak
| HardBreak
| CodeSpan { code: string }
| Emphasis { children: Inline[] }
| Strong { children: Inline[] }
| Strikethrough { children: Inline[] }
| Link { url: string, title: string?, children: Inline[] }
| Picture { url: string, title: string?, alt: Inline[] }
| HtmlInline { html: string }
/// What happens to raw HTML in the source:
/// - `"escape"` (the default) shows it as text;
/// - `"sanitize"` keeps what the `policy` allows and removes the rest;
/// - `"trust"` passes it through. Use it only for Markdown you wrote yourself.
export type HtmlMode = "escape" | "sanitize" | "trust"
/// What `sanitize` keeps. Anything left out takes the default: common formatting tags, no scripts, styles, event
/// handlers or forms.
export type Policy = {
/// Lowercase tag names to keep. A tag not listed is removed and its content kept.
tags?: string[],
/// Attributes to keep, per lowercase tag name.
attributes?: Map<string, string[]>,
/// URL schemes `href` and `src` may use. Relative URLs are always allowed.
schemes?: string[],
}
/// Whether a URL is a link's or an image's.
export type UrlKind = "link" | "image"
/// Options for parsing and rendering. Every field is optional.
export type Options = {
/// GitHub Flavored Markdown: tables, task lists, strikethrough and bare autolinks. On by default.
gfm?: boolean,
/// What happens to raw HTML. `"escape"` by default.
html?: HtmlMode,
/// What `html: "sanitize"` keeps.
policy?: Policy,
/// URL schemes links and images may use: `http`, `https`, `mailto` and `tel` by default. Relative URLs are always
/// allowed, and images may also be `data:image/` PNG, GIF, JPEG or WebP. A link to anything else renders as its text.
schemes?: string[],
/// Added to every heading's level, up to 6: `1` renders `# Title` as `<h2>`.
headingOffset?: float,
/// `target` for every link. `"_blank"` also adds `rel="noopener noreferrer"`.
linkTarget?: string,
/// Rewrites a URL before it is checked and rendered, as an app resolves `upload:<id>` to a real address.
/// Returning `null` keeps the URL as written.
rewriteUrl?: (url: string, kind: UrlKind) => string?,
}

12
markdown/bench/bench.weft Normal file
View file

@ -0,0 +1,12 @@
// `weft run bench/bench.weft <copies>`: renders the CommonMark spec's examples, joined and repeated, and prints the size.
// scripts/bench.mjs times it from outside the VM, beside markdown-it on the same text.
import { args } from "std/process"
import { log } from "std/log"
import { toHtml } from "../markdown"
import { COMMONMARK } from "../tests/cases"
let copies = try int(args().at(0) ?? "1") else 1
let text = COMMONMARK.map(c => c.markdown).join("\n\n").repeat(copies)
if copies > 0 {
log(toHtml(text).length)
}

1145
markdown/lib/blocks.weft Normal file

File diff suppressed because it is too large Load diff

242
markdown/lib/chars.weft Normal file
View file

@ -0,0 +1,242 @@
// Character classes and the text transformations CommonMark defines: escapes, entities, URLs.
import { entity } from "./entities"
import { PUNCTUATION } from "./unicode"
/// The code unit at `i`, or -1 past the end.
export fn unit(s: string, i: int): int {
return s.codeUnitAt(i) ?? -1
}
export fn isSpaceOrTab(c: float): boolean {
return c == 32 || c == 9
}
export fn isDigit(c: float): boolean {
return c >= 48 && c <= 57
}
export fn isAlpha(c: float): boolean {
return (c >= 65 && c <= 90) || (c >= 97 && c <= 122)
}
export fn isAlnum(c: float): boolean {
return isAlpha(c) || isDigit(c)
}
export fn isHex(c: float): boolean {
return isDigit(c) || (c >= 65 && c <= 70) || (c >= 97 && c <= 102)
}
/// ASCII punctuation: what a backslash can escape.
export fn isAsciiPunct(c: float): boolean {
return (c >= 33 && c <= 47) ||
(c >= 58 && c <= 64) ||
(c >= 91 && c <= 96) ||
(c >= 123 && c <= 126)
}
/// Unicode whitespace: the `Zs` category, tab, line feed, form feed and carriage return.
export fn isUnicodeSpace(c: float): boolean {
if c < 128 {
return c == 32 || c == 9 || c == 10 || c == 12 || c == 13
}
return c == 0xa0 ||
c == 0x1680 ||
(c >= 0x2000 && c <= 0x200a) ||
c == 0x202f ||
c == 0x205f ||
c == 0x3000
}
/// Unicode punctuation: the `P` and `S` categories.
export fn isUnicodePunct(c: float): boolean {
if c < 128 {
return isAsciiPunct(c)
}
let mut lo = 0
let mut hi = PUNCTUATION.length.div(2) - 1
while lo <= hi {
let mid = (lo + hi) >> 1
if c < PUNCTUATION[mid * 2] {
hi = mid - 1
} else if c > PUNCTUATION[mid * 2 + 1] {
lo = mid + 1
} else {
return true
}
}
return false
}
/// Whether `s` holds only spaces and tabs (or nothing).
export fn isBlank(s: string): boolean {
for i of 0..s.length {
let c = s.codeUnitAt(i) ?? 32
if c != 32 && c != 9 && c != 10 && c != 13 {
return false
}
}
return true
}
/// The code point ending just before `pos`, or a line feed at the start of `s`.
export fn codePointBefore(s: string, pos: int): int {
if pos <= 0 {
return 10
}
let low = s.codeUnitAt(pos - 1) ?? 10
if low >= 0xdc00 && low <= 0xdfff && pos >= 2 {
let high = s.codeUnitAt(pos - 2) ?? 0
if high >= 0xd800 && high <= 0xdbff {
return (high - 0xd800) * 0x400 + (low - 0xdc00) + 0x10000
}
}
return low
}
/// The character reference starting at the `&` at `pos`: its text and where it ends, or `null` when there is none.
export fn scanEntity(s: string, pos: int): [string, int]? {
let mut i = pos + 1
if s.codeUnitAt(i) == 35 {
i += 1
let c = s.codeUnitAt(i) ?? -1
let hex = c == 120 || c == 88
if hex {
i += 1
}
let start = i
while i - start < (hex ? 6 : 7) &&
(hex ? isHex(s.codeUnitAt(i) ?? -1) : isDigit(s.codeUnitAt(i) ?? -1)) {
i += 1
}
if i == start || s.codeUnitAt(i) != 59 {
return null
}
let n = try int(s.slice(start, i), hex ? 16 : 10) else 0
let valid = n != 0 && n <= 0x10ffff && !(n >= 0xd800 && n <= 0xdfff)
return [String.fromCodePoint(valid ? n : 0xfffd), i + 1]
}
let start = i
while i - start < 32 && isAlnum(s.codeUnitAt(i) ?? -1) {
i += 1
}
if i == start || s.codeUnitAt(i) != 59 {
return null
}
let text = entity(s.slice(start, i))
if text == null {
return null
}
return [text, i + 1]
}
/// `s` with its backslash escapes and character references replaced by what they stand for.
export fn unescape(s: string): string {
if !s.includes("\\") && !s.includes("&") {
return s
}
let mut parts: string[] = []
let mut from = 0
let mut i = 0
while i < s.length {
let c = s.codeUnitAt(i) ?? 0
if c == 92 && isAsciiPunct(s.codeUnitAt(i + 1) ?? 0) {
parts.push(s.slice(from, i))
from = i + 1
i += 2
} else if c == 38 {
let found = scanEntity(s, i)
if found != null {
parts.push(s.slice(from, i))
parts.push(found[0])
i = found[1]
from = i
} else {
i += 1
}
} else {
i += 1
}
}
parts.push(s.slice(from))
return parts.join("")
}
/// `s` escaped for HTML text and attribute values.
export fn escapeHtml(s: string): string {
let mut i = 0
while i < s.length {
let c = s.codeUnitAt(i) ?? 0
if c == 38 || c == 60 || c == 62 || c == 34 {
break
}
i += 1
}
if i == s.length {
return s
}
let mut parts: string[] = [s.slice(0, i)]
let mut from = i
while i < s.length {
let c = s.codeUnitAt(i) ?? 0
let rep = c == 38 ? "&amp;" : c == 60 ? "&lt;" : c == 62 ? "&gt;" : c == 34 ? "&quot;" : ""
if rep != "" {
parts.push(s.slice(from, i))
parts.push(rep)
from = i + 1
}
i += 1
}
parts.push(s.slice(from))
return parts.join("")
}
fn keptInUrl(c: int): boolean {
// Letters, digits and what URLs use as syntax; everything else is percent-encoded.
return isAlnum(c) || (c < 128 && ";/?:@&=+$,-_.!~*'()#".includes(String.fromCharCode(c)))
}
/// `url` percent-encoded for an `href`: what is already encoded stays, and a lone surrogate becomes U+FFFD.
export fn normalizeUrl(url: string): string {
let mut parts: string[] = []
let mut from = 0
let mut i = 0
while i < url.length {
let c = url.codeUnitAt(i) ?? 0
if c == 37 && isHex(url.codeUnitAt(i + 1) ?? -1) && isHex(url.codeUnitAt(i + 2) ?? -1) {
i += 3
} else if keptInUrl(c) {
i += 1
} else {
parts.push(url.slice(from, i))
let point = url.codePointAt(i) ?? 0xfffd
let width = point > 0xffff ? 2 : 1
let safe = point >= 0xd800 && point <= 0xdfff ? 0xfffd : point
for b of Bytes.fromUtf8(String.fromCodePoint(safe)).toList() {
parts.push("%" + str(b, { radix: 16 }).toUpperCase().padStart(2, "0"))
}
i += width
from = i
}
}
parts.push(url.slice(from))
return parts.join("")
}
/// A link label as references are matched: case-folded, with whitespace collapsed.
export fn normalizeLabel(label: string): string {
let mut out: string[] = []
let mut space = false
for ch of label.trim() {
if ch == " " || ch == "\t" || ch == "\n" || ch == "\r" {
space = true
} else {
if space {
out.push(" ")
space = false
}
out.push(ch)
}
}
return out.join("").toLowerCase().toUpperCase()
}

File diff suppressed because one or more lines are too long

1397
markdown/lib/inlines.weft Normal file

File diff suppressed because it is too large Load diff

255
markdown/lib/render.weft Normal file
View file

@ -0,0 +1,255 @@
// The HTML renderer: the output CommonMark's and GFM's spec tests expect.
import type {
Align,
Block,
Document,
HtmlMode,
Inline,
ListItem,
Options,
TableCell,
UrlKind,
} from "../ast"
import { escapeHtml } from "./chars"
import { DEFAULT_SCHEMES, isSafeUrl, sanitize } from "./sanitize"
type Out = {
parts: string[],
atLineStart: boolean,
mode: HtmlMode,
schemes: string[],
offset: float,
target: string?,
}
fn put(mut o: Out, text: string) {
if text != "" {
o.parts.push(text)
o.atLineStart = text.endsWith("\n")
}
}
fn targetOf(o: Out): string? {
return o.target
}
fn isEscaping(o: Out): boolean {
return o.mode == "escape"
}
fn isAtLineStart(o: Out): boolean {
return o.atLineStart
}
fn cr(mut o: Out) {
if !isAtLineStart(o) {
put(&o, "\n")
}
}
fn rawHtml(o: Out, html: string): string {
return o.mode == "escape" ? escapeHtml(html) : html
}
fn plainText(inlines: Inline[]): string {
let mut out: string[] = []
for n of inlines {
match n {
Str { text } => out.push(text),
SoftBreak | HardBreak => out.push(" "),
CodeSpan { code } => out.push(code),
Emphasis { children } | Strong { children } | Strikethrough { children } => out.push(
plainText(children),
),
Link { children } => out.push(plainText(children)),
Picture { alt } => out.push(plainText(alt)),
HtmlInline { html } => out.push(html),
}
}
return out.join("")
}
fn wrap(mut o: Out, tag: string, children: Inline[], options: Options): void {
put(&o, `<${tag}>`)
renderInlines(&o, children, options)
put(&o, `</${tag}>`)
}
/// The URL to render, or `null` when it isn't safe to.
fn finalUrl(url: string, kind: UrlKind, o: Out, options: Options): string? {
let rewrite = options.rewriteUrl
let rewritten = rewrite != null ? rewrite(url, kind) ?? url : url
return o.mode == "trust" || isSafeUrl(rewritten, o.schemes, kind == "image") ? rewritten : null
}
fn renderInlines(mut o: Out, inlines: Inline[], options: Options): void {
for n of inlines {
match n {
Str { text } => put(&o, escapeHtml(text)),
SoftBreak => put(&o, "\n"),
HardBreak => put(&o, "<br />\n"),
CodeSpan { code } => put(&o, `<code>${escapeHtml(code)}</code>`),
Emphasis { children } => wrap(&o, "em", children, options),
Strong { children } => wrap(&o, "strong", children, options),
Strikethrough { children } => wrap(&o, "del", children, options),
HtmlInline { html } => {
let raw = rawHtml(o, html)
put(&o, raw)
},
Link { url, title, children } => {
let href = finalUrl(url, "link", o, options)
if href == null {
renderInlines(&o, children, options)
} else {
let titled = title != null && title != "" ? ` title="${escapeHtml(title)}"` : ""
let linkTarget = targetOf(o)
let target = linkTarget != null ? ` target="${escapeHtml(linkTarget)}"` : ""
let rel = linkTarget == "_blank" ? " rel=\"noopener noreferrer\"" : ""
put(&o, `<a href="${escapeHtml(href)}"${titled}${target}${rel}>`)
renderInlines(&o, children, options)
put(&o, "</a>")
}
},
Picture { url, title, alt } => {
let src = finalUrl(url, "image", o, options)
if src == null {
put(&o, escapeHtml(plainText(alt)))
} else {
let titled = title != null && title != "" ? ` title="${escapeHtml(title)}"` : ""
put(&o, `<img src="${escapeHtml(src)}" alt="${escapeHtml(plainText(alt))}"${titled} />`)
}
},
}
}
}
fn renderItem(mut o: Out, item: ListItem, tight: boolean, options: Options): void {
put(&o, "<li>")
if item.checked != null {
put(
&o,
item.checked
? "<input checked=\"\" disabled=\"\" type=\"checkbox\"> "
: "<input disabled=\"\" type=\"checkbox\"> ",
)
}
renderBlocks(&o, item.children, tight, options)
put(&o, "</li>")
cr(&o)
}
fn renderCells(mut o: Out, tag: string, cells: TableCell[], align: Align[], options: Options) {
put(&o, "<tr>\n")
for [i, cell] of cells.entries() {
let a = align.at(i) ?? null
put(&o, a != null ? `<${tag} align="${a}">` : `<${tag}>`)
renderInlines(&o, cell.children, options)
put(&o, `</${tag}>\n`)
}
put(&o, "</tr>\n")
}
fn renderBlocks(mut o: Out, blocks: Block[], tight: boolean, options: Options): void {
for block of blocks {
match block {
Paragraph { children } => {
if tight {
renderInlines(&o, children, options)
} else {
cr(&o)
wrap(&o, "p", children, options)
cr(&o)
}
},
Heading { level, children } => {
let shifted = level + o.offset
let shown = shifted > 6 ? 6 : shifted < 1 ? 1 : shifted
cr(&o)
wrap(&o, `h${shown}`, children, options)
cr(&o)
},
ThematicBreak => {
cr(&o)
put(&o, "<hr />")
cr(&o)
},
CodeBlock { info, code } => {
cr(&o)
let language = info.trim().split(" ")[0].split("\t")[0]
put(
&o,
language != "" ? `<pre><code class="language-${escapeHtml(language)}">` : "<pre><code>",
)
put(&o, escapeHtml(code))
put(&o, "</code></pre>")
cr(&o)
},
HtmlBlock { html } => {
cr(&o)
if isEscaping(o) {
put(&o, `<p>${escapeHtml(html)}</p>`)
} else {
put(&o, html)
}
cr(&o)
},
Blockquote { children } => {
cr(&o)
put(&o, "<blockquote>\n")
renderBlocks(&o, children, false, options)
cr(&o)
put(&o, "</blockquote>")
cr(&o)
},
BulletList { tight: itemsTight, items } => {
cr(&o)
put(&o, "<ul>\n")
for item of items {
renderItem(&o, item, itemsTight, options)
}
put(&o, "</ul>")
cr(&o)
},
OrderedList { tight: itemsTight, start, items } => {
cr(&o)
put(&o, start != 1 ? `<ol start="${start}">\n` : "<ol>\n")
for item of items {
renderItem(&o, item, itemsTight, options)
}
put(&o, "</ol>")
cr(&o)
},
Table { align, head, rows } => {
cr(&o)
put(&o, "<table>\n<thead>\n")
renderCells(&o, "th", head, align, options)
put(&o, "</thead>\n")
if rows.length > 0 {
put(&o, "<tbody>\n")
for row of rows {
renderCells(&o, "td", row, align, options)
}
put(&o, "</tbody>\n")
}
put(&o, "</table>")
cr(&o)
},
}
}
}
/// Renders a document to HTML.
export fn renderDocument(doc: Document, options: Options): string {
let mode = options.html ?? "escape"
let mut o: Out = {
parts: [],
atLineStart: true,
mode,
schemes: options.schemes ?? DEFAULT_SCHEMES,
offset: options.headingOffset ?? 0,
target: options.linkTarget,
}
renderBlocks(&o, doc.children, false, options)
let html = o.parts.join("")
return mode == "sanitize" ? sanitize(html, options.policy) : html
}

403
markdown/lib/sanitize.weft Normal file
View file

@ -0,0 +1,403 @@
// The HTML sanitiser: an allowlist over a tokenizer. What comes out is well formed and holds only allowed tags and
// attributes, every text escaped, so nothing a browser parses differently can bring a removed element back.
import type { Policy } from "../ast"
import { escapeHtml, isAlpha, scanEntity } from "./chars"
let DEFAULT_TAGS = [
"a",
"abbr",
"b",
"blockquote",
"br",
"code",
"dd",
"del",
"details",
"div",
"dl",
"dt",
"em",
"figcaption",
"figure",
"h1",
"h2",
"h3",
"h4",
"h5",
"h6",
"hr",
"i",
"img",
"input",
"ins",
"kbd",
"li",
"mark",
"ol",
"p",
"pre",
"q",
"s",
"samp",
"small",
"span",
"strong",
"sub",
"summary",
"sup",
"table",
"tbody",
"td",
"tfoot",
"th",
"thead",
"tr",
"u",
"ul",
"var",
]
let DEFAULT_ATTRIBUTES = Map<string, string[]>(
[
["a", ["href", "title", "target"]],
["img", ["src", "alt", "title", "width", "height"]],
["ol", ["start"]],
["code", ["class"]],
["td", ["align", "colspan", "rowspan"]],
["th", ["align", "colspan", "rowspan", "scope"]],
["input", ["type", "checked", "disabled"]],
["details", ["open"]],
],
)
/// URL schemes allowed unless the options say otherwise.
export let DEFAULT_SCHEMES = ["http", "https", "mailto", "tel"]
let VOID_TAGS = Set(
[
"br",
"hr",
"img",
"input",
"wbr",
"area",
"base",
"col",
"embed",
"link",
"meta",
"param",
"source",
"track",
],
)
/// Elements removed with everything inside them.
let DROPPED = Set(
[
"script",
"style",
"template",
"iframe",
"object",
"embed",
"applet",
"noscript",
"noembed",
"noframes",
"title",
"textarea",
"xmp",
"plaintext",
"svg",
"math",
"select",
"frameset",
"head",
],
)
fn isDataImage(url: string): boolean {
let lower = url.slice(0, 16).toLowerCase()
return lower.startsWith("data:image/png") ||
lower.startsWith("data:image/gif") ||
lower.startsWith("data:image/jpeg") ||
lower.startsWith("data:image/jpg") ||
lower.startsWith("data:image/webp")
}
/// Whether a URL is relative or uses one of `schemes`. Browsers ignore tabs, line breaks and leading control characters
/// in a scheme, so they are ignored here too.
export fn isSafeUrl(url: string, schemes: string[], image: boolean): boolean {
let mut compact: string[] = []
for ch of url {
let c = ch.codeUnitAt(0) ?? 0
if c > 32 && c != 127 {
compact.push(ch)
}
}
let text = compact.join("")
let mut colon = -1
for i of 0..text.length {
let c = text.codeUnitAt(i) ?? 0
if c == 58 {
colon = i
break
}
if c == 47 || c == 63 || c == 35 {
return true
}
}
if colon < 0 {
return true
}
if image && isDataImage(text) {
return true
}
return schemes.includes(text.slice(0, colon).toLowerCase())
}
/// Text decoded from an attribute value: its character references replaced.
fn decodeAttribute(value: string): string {
if !value.includes("&") {
return value
}
let mut parts: string[] = []
let mut from = 0
let mut i = value.indexOf("&")
while i >= 0 {
let found = scanEntity(value, i)
if found != null {
parts.push(value.slice(from, i))
parts.push(found[0])
from = found[1]
i = value.indexOf("&", from)
} else {
i = value.indexOf("&", i + 1)
}
}
parts.push(value.slice(from))
return parts.join("")
}
/// Text as it may appear between tags: a character reference stays, anything else that means something is escaped.
fn cleanText(text: string): string {
if !text.includes("&") && !text.includes("<") && !text.includes(">") {
return text
}
let mut parts: string[] = []
let mut from = 0
let mut i = 0
while i < text.length {
let c = text.codeUnitAt(i) ?? 0
if c == 38 {
let found = scanEntity(text, i)
if found != null {
i = found[1]
} else {
parts.push(text.slice(from, i))
parts.push("&amp;")
i += 1
from = i
}
} else if c == 60 || c == 62 {
parts.push(text.slice(from, i))
parts.push(c == 60 ? "&lt;" : "&gt;")
i += 1
from = i
} else {
i += 1
}
}
parts.push(text.slice(from))
return parts.join("")
}
type Attr = { name: string, value: string }
type Tag = { name: string, attrs: Attr[], end: int }
fn isSpace(c: int): boolean {
return c == 32 || c == 9 || c == 10 || c == 12 || c == 13
}
/// Reads the tag starting at `<` at `pos` (its name begins at `start`).
fn readTag(html: string, start: int): Tag {
let mut i = start
while i < html.length {
let c = html.codeUnitAt(i) ?? 32
if isSpace(c) || c == 62 || c == 47 {
break
}
i += 1
}
let name = html.slice(start, i).toLowerCase()
let mut attrs: Attr[] = []
while i < html.length {
let c = html.codeUnitAt(i) ?? 32
if c == 62 {
return { name, attrs, end: i + 1 }
}
if isSpace(c) || c == 47 {
i += 1
continue
}
let nameStart = i
i += 1
while i < html.length {
let u = html.codeUnitAt(i) ?? 32
if isSpace(u) || u == 61 || u == 62 || u == 47 {
break
}
i += 1
}
let attr = html.slice(nameStart, i).toLowerCase()
while isSpace(html.codeUnitAt(i) ?? 0) {
i += 1
}
let mut value = ""
if html.codeUnitAt(i) == 61 {
i += 1
while isSpace(html.codeUnitAt(i) ?? 0) {
i += 1
}
let q = html.codeUnitAt(i) ?? -1
if q == 34 || q == 39 {
let close = html.indexOf(String.fromCharCode(q), i + 1)
let stop = close < 0 ? html.length : close
value = html.slice(i + 1, stop)
i = close < 0 ? html.length : close + 1
} else {
let from = i
while i < html.length {
let u = html.codeUnitAt(i) ?? 32
if isSpace(u) || u == 62 {
break
}
i += 1
}
value = html.slice(from, i)
}
}
attrs.push({ name: attr, value: decodeAttribute(value) })
}
return { name, attrs, end: html.length }
}
/// Where the content of the dropped element `name` ends: after its closing tag, or at the end of the text.
fn skipElement(html: string, name: string, from: int): int {
let mut i = html.indexOf("</", from)
while i >= 0 {
if html.slice(i + 2, i + 2 + name.length).toLowerCase() == name {
let after = html.codeUnitAt(i + 2 + name.length) ?? 62
if isSpace(after) || after == 62 || after == 47 {
let close = html.indexOf(">", i)
return close < 0 ? html.length : close + 1
}
}
i = html.indexOf("</", i + 2)
}
return html.length
}
fn keepAttribute(tag: string, a: Attr, schemes: string[]): boolean {
if a.name == "href" || a.name == "src" {
return isSafeUrl(a.value, schemes, tag == "img")
}
if a.name == "target" {
return a.value == "_blank"
}
if a.name == "type" && tag == "input" {
return a.value.toLowerCase() == "checkbox"
}
if a.name == "class" {
return a.value.startsWith("language-") && !a.value.includes(" ")
}
return true
}
/// The only input kept is a task list's checkbox, always disabled: a text field in rendered content can phish.
fn isCheckbox(tag: Tag): boolean {
return tag.attrs.some(a => a.name == "type" && a.value.toLowerCase() == "checkbox")
}
fn openTag(tag: Tag, allowed: string[], schemes: string[]): string {
let mut out: string[] = ["<", tag.name]
let mut blank = false
let mut seen = Set<string>()
for a of tag.attrs {
if allowed.includes(a.name) && !seen.has(a.name) && keepAttribute(tag.name, a, schemes) {
seen.add(a.name)
out.push(` ${a.name}="${escapeHtml(a.value)}"`)
blank = blank || (a.name == "target" && a.value == "_blank")
}
}
if blank {
out.push(" rel=\"noopener noreferrer\"")
}
if tag.name == "input" && !seen.has("disabled") {
out.push(" disabled=\"\"")
}
out.push(VOID_TAGS.has(tag.name) ? " />" : ">")
return out.join("")
}
/// Removes everything `policy` doesn't allow from `html`: scripts, styles, event handlers and unsafe URLs by default.
/// The result is well formed: stray closing tags are dropped and open ones closed.
///
/// ```weft
/// sanitize("<p onclick=\"x()\">hi<script>bad()</script></p>") // "<p>hi</p>"
/// ```
export fn sanitize(html: string, policy?: Policy): string {
let tags = Set(policy?.tags ?? DEFAULT_TAGS)
let attributes = policy?.attributes ?? DEFAULT_ATTRIBUTES
let schemes = policy?.schemes ?? DEFAULT_SCHEMES
let mut out: string[] = []
let mut open: string[] = []
let mut i = 0
while i < html.length {
let lt = html.indexOf("<", i)
if lt < 0 {
out.push(cleanText(html.slice(i)))
break
}
if lt > i {
out.push(cleanText(html.slice(i, lt)))
}
let next = html.codeUnitAt(lt + 1) ?? -1
if html.startsWith("<!--", lt) {
let end = html.indexOf("-->", lt + 4)
i = end < 0 ? html.length : end + 3
} else if next == 33 || next == 63 {
let end = html.indexOf(">", lt + 2)
i = end < 0 ? html.length : end + 1
} else if next == 47 && isAlpha(html.codeUnitAt(lt + 2) ?? -1) {
let tag = readTag(html, lt + 2)
i = tag.end
let at = open.lastIndexOf(tag.name)
if at >= 0 {
while open.length > at {
out.push(`</${open.pop() ?? ""}>`)
}
}
} else if isAlpha(next) {
let tag = readTag(html, lt + 1)
i = tag.end
if DROPPED.has(tag.name) {
i = VOID_TAGS.has(tag.name) ? i : skipElement(html, tag.name, i)
} else if tags.has(tag.name) && (tag.name != "input" || isCheckbox(tag)) {
out.push(openTag(tag, attributes.get(tag.name) ?? [], schemes))
if !VOID_TAGS.has(tag.name) {
open.push(tag.name)
}
}
} else {
out.push("&lt;")
i = lt + 1
}
}
while open.length > 0 {
out.push(`</${open.pop() ?? ""}>`)
}
return out.join("")
}

50
markdown/lib/unicode.weft Normal file
View file

@ -0,0 +1,50 @@
// Generated by scripts/gen-tables.mjs from the engine's Unicode tables (general categories P and S). Don't edit it.
/// Sorted, inclusive ranges of code points: start, end, start, end, …
export let PUNCTUATION: int[] = [
33, 47, 58, 64, 91, 96, 123, 126, 161, 169, 171, 172, 174, 177, 180, 180,
182, 184, 187, 187, 191, 191, 215, 215, 247, 247, 706, 709, 722, 735, 741, 747,
749, 749, 751, 767, 885, 885, 894, 894, 900, 901, 903, 903, 1014, 1014, 1154, 1154,
1370, 1375, 1417, 1418, 1421, 1423, 1470, 1470, 1472, 1472, 1475, 1475, 1478, 1478, 1523, 1524,
1542, 1551, 1563, 1563, 1565, 1567, 1642, 1645, 1748, 1748, 1758, 1758, 1769, 1769, 1789, 1790,
1792, 1805, 2038, 2041, 2046, 2047, 2096, 2110, 2142, 2142, 2184, 2184, 2404, 2405, 2416, 2416,
2546, 2547, 2554, 2555, 2557, 2557, 2678, 2678, 2800, 2801, 2928, 2928, 3059, 3066, 3191, 3191,
3199, 3199, 3204, 3204, 3407, 3407, 3449, 3449, 3572, 3572, 3647, 3647, 3663, 3663, 3674, 3675,
3841, 3863, 3866, 3871, 3892, 3892, 3894, 3894, 3896, 3896, 3898, 3901, 3973, 3973, 4030, 4037,
4039, 4044, 4046, 4058, 4170, 4175, 4254, 4255, 4347, 4347, 4960, 4968, 5008, 5017, 5120, 5120,
5741, 5742, 5787, 5788, 5867, 5869, 5941, 5942, 6100, 6102, 6104, 6107, 6144, 6154, 6464, 6464,
6468, 6469, 6622, 6655, 6686, 6687, 6816, 6822, 6824, 6829, 6990, 6991, 7002, 7018, 7028, 7039,
7164, 7167, 7227, 7231, 7294, 7295, 7360, 7367, 7379, 7379, 8125, 8125, 8127, 8129, 8141, 8143,
8157, 8159, 8173, 8175, 8189, 8190, 8208, 8231, 8240, 8286, 8314, 8318, 8330, 8334, 8352, 8385,
8448, 8449, 8451, 8454, 8456, 8457, 8468, 8468, 8470, 8472, 8478, 8483, 8485, 8485, 8487, 8487,
8489, 8489, 8494, 8494, 8506, 8507, 8512, 8516, 8522, 8525, 8527, 8527, 8586, 8587, 8592, 9257,
9280, 9290, 9372, 9449, 9472, 10101, 10132, 11123, 11126, 11263, 11493, 11498, 11513, 11516, 11518, 11519,
11632, 11632, 11776, 11822, 11824, 11869, 11904, 11929, 11931, 12019, 12032, 12245, 12272, 12287, 12289, 12292,
12296, 12320, 12336, 12336, 12342, 12343, 12349, 12351, 12443, 12444, 12448, 12448, 12539, 12539, 12688, 12689,
12694, 12703, 12736, 12773, 12783, 12783, 12800, 12830, 12842, 12871, 12880, 12880, 12896, 12927, 12938, 12976,
12992, 13311, 19904, 19967, 42128, 42182, 42238, 42239, 42509, 42511, 42611, 42611, 42622, 42622, 42738, 42743,
42752, 42774, 42784, 42785, 42889, 42890, 43048, 43051, 43062, 43065, 43124, 43127, 43214, 43215, 43256, 43258,
43260, 43260, 43310, 43311, 43359, 43359, 43457, 43469, 43486, 43487, 43612, 43615, 43639, 43641, 43742, 43743,
43760, 43761, 43867, 43867, 43882, 43883, 44011, 44011, 64297, 64297, 64434, 64466, 64830, 64847, 64912, 64913,
64968, 64975, 65020, 65023, 65040, 65049, 65072, 65106, 65108, 65126, 65128, 65131, 65281, 65295, 65306, 65312,
65339, 65344, 65371, 65381, 65504, 65510, 65512, 65518, 65532, 65533, 65792, 65794, 65847, 65855, 65913, 65929,
65932, 65934, 65936, 65948, 65952, 65952, 66000, 66044, 66463, 66463, 66512, 66512, 66927, 66927, 67671, 67671,
67703, 67704, 67871, 67871, 67903, 67903, 68176, 68184, 68223, 68223, 68296, 68296, 68336, 68342, 68409, 68415,
68505, 68508, 68974, 68974, 69006, 69007, 69293, 69293, 69328, 69336, 69461, 69465, 69510, 69513, 69703, 69709,
69819, 69820, 69822, 69825, 69952, 69955, 70004, 70005, 70085, 70088, 70093, 70093, 70107, 70107, 70109, 70111,
70200, 70205, 70313, 70313, 70612, 70613, 70615, 70616, 70731, 70735, 70746, 70747, 70749, 70749, 70854, 70854,
71105, 71127, 71233, 71235, 71264, 71276, 71353, 71353, 71484, 71487, 71739, 71739, 72004, 72006, 72162, 72162,
72255, 72262, 72346, 72348, 72350, 72354, 72448, 72457, 72673, 72673, 72769, 72773, 72816, 72817, 73463, 73464,
73539, 73551, 73685, 73713, 73727, 73727, 74864, 74868, 77809, 77810, 92782, 92783, 92917, 92917, 92983, 92991,
92996, 92997, 93549, 93551, 93847, 93850, 94178, 94178, 113820, 113820, 113823, 113823, 117760, 117999, 118010, 118012,
118016, 118451, 118458, 118480, 118496, 118512, 118608, 118723, 118784, 119029, 119040, 119078, 119081, 119140, 119146, 119148,
119171, 119172, 119180, 119209, 119214, 119274, 119296, 119361, 119365, 119365, 119552, 119638, 120513, 120513, 120539, 120539,
120571, 120571, 120597, 120597, 120629, 120629, 120655, 120655, 120687, 120687, 120713, 120713, 120745, 120745, 120771, 120771,
120832, 121343, 121399, 121402, 121453, 121460, 121462, 121475, 121477, 121483, 123215, 123215, 123647, 123647, 124415, 124415,
125278, 125279, 126124, 126124, 126128, 126128, 126254, 126254, 126704, 126705, 126976, 127019, 127024, 127123, 127136, 127150,
127153, 127167, 127169, 127183, 127185, 127221, 127245, 127405, 127462, 127490, 127504, 127547, 127552, 127560, 127568, 127569,
127584, 127589, 127744, 128728, 128732, 128748, 128752, 128764, 128768, 128985, 128992, 129003, 129008, 129008, 129024, 129035,
129040, 129095, 129104, 129113, 129120, 129159, 129168, 129197, 129200, 129211, 129216, 129217, 129232, 129240, 129280, 129623,
129632, 129645, 129648, 129660, 129664, 129674, 129678, 129734, 129736, 129736, 129741, 129756, 129759, 129770, 129775, 129784,
129792, 129938, 129940, 130031, 130042, 130042,
]

156
markdown/markdown.weft Normal file
View file

@ -0,0 +1,156 @@
// `@weft/markdown`: CommonMark and GitHub Flavored Markdown.
import type { Block, Document, Inline, Options, Policy } from "./ast"
import { parseDocument } from "./lib/blocks"
import { renderDocument } from "./lib/render"
import { sanitize as clean } from "./lib/sanitize"
/// Parses Markdown into its syntax tree. Parsing never fails: any text is a document.
///
/// ```weft
/// let doc = parse("# Hello\n\nSome *text*.")
/// doc.children.length // 2
/// ```
export fn parse(source: string, options: Options = {}): Document {
return parseDocument(source, options.gfm ?? true)
}
/// Renders a syntax tree to HTML. Raw HTML is escaped and unsafe URLs are removed unless `options` say otherwise.
export fn render(doc: Document, options: Options = {}): string {
return renderDocument(doc, options)
}
/// Markdown to HTML, safe for untrusted input by default.
///
/// ```weft
/// toHtml("**bold** <script>alert(1)</script>")
/// // "<p><strong>bold</strong> &lt;script&gt;alert(1)&lt;/script&gt;</p>\n"
///
/// toHtml("# Title", { headingOffset: 1, linkTarget: "_blank" })
/// // "<h2>Title</h2>\n"
/// ```
export fn toHtml(source: string, options: Options = {}): string {
return renderDocument(parseDocument(source, options.gfm ?? true), options)
}
/// Removes everything `policy` doesn't allow from HTML, whatever produced it. See `Policy` for the defaults.
///
/// ```weft
/// sanitize("<a href=\"javascript:x()\" title=\"t\">hi</a>") // "<a title=\"t\">hi</a>"
/// ```
export fn sanitize(html: string, policy?: Policy): string {
return clean(html, policy)
}
fn inlineText(inlines: Inline[]): string {
let mut out: string[] = []
for n of inlines {
match n {
Str { text } => out.push(text),
SoftBreak => out.push(" "),
HardBreak => out.push("\n"),
CodeSpan { code } => out.push(code),
Emphasis { children } | Strong { children } | Strikethrough { children } => out.push(
inlineText(children),
),
Link { url, children } => {
let label = inlineText(children)
out.push(label == url || `mailto:${label}` == url ? label : `${label} (${url})`)
},
Picture { alt } => {
let label = inlineText(alt)
out.push(label == "" ? "[image]" : `[image: ${label}]`)
},
HtmlInline => {},
}
}
return out.join("")
}
fn blockText(blocks: Block[], mut out: string[]): void {
for block of blocks {
match block {
Paragraph { children } | Heading { children } => out.push(inlineText(children)),
ThematicBreak => out.push("---"),
CodeBlock { code } => out.push(code.trimEnd()),
HtmlBlock => {},
Blockquote { children } => blockText(children, &out),
BulletList { items } | OrderedList { items } => {
for item of items {
let mut inner: string[] = []
blockText(item.children, &inner)
let mark = item.checked == null ? "- " : item.checked ? "- [x] " : "- [ ] "
out.push(mark + inner.join("\n "))
}
},
Table { head, rows } => {
out.push(head.map(c => inlineText(c.children)).join(" | "))
for row of rows {
out.push(row.map(c => inlineText(c.children)).join(" | "))
}
},
}
}
}
/// The document as plain text, for previews, search and the text part of an email. Links keep their address and images
/// become `[image: caption]`.
///
/// ```weft
/// toText(parse("Read [the guide](https://weft.dev)."))
/// // "Read the guide (https://weft.dev)."
/// ```
export fn toText(doc: Document): string {
let mut out: string[] = []
blockText(doc.children, &out)
return out.join("\n\n")
}
fn visitInlines(inlines: Inline[], f: (node: Inline) => void): void {
for n of inlines {
f(n)
match n {
Emphasis { children } | Strong { children } | Strikethrough { children } | Link {
children,
} => visitInlines(children, f),
Picture { alt } => visitInlines(alt, f),
_ => {},
}
}
}
fn visitBlocks(blocks: Block[], f: (node: Inline) => void): void {
for block of blocks {
match block {
Paragraph { children } | Heading { children } => visitInlines(children, f),
Blockquote { children } => visitBlocks(children, f),
BulletList { items } | OrderedList { items } => {
for item of items {
visitBlocks(item.children, f)
}
},
Table { head, rows } => {
for cell of head {
visitInlines(cell.children, f)
}
for row of rows {
for cell of row {
visitInlines(cell.children, f)
}
}
},
_ => {},
}
}
}
/// Calls `f` with every inline of the document, in order, outermost first.
///
/// ```weft
/// let mut images: string[] = []
/// visit(parse(source), n => {
/// if let Picture { url } = n { images.push(url) }
/// })
/// ```
export fn visit(doc: Document, f: (node: Inline) => void) {
visitBlocks(doc.children, f)
}

View file

@ -0,0 +1,49 @@
// Times @weft/markdown on the VM against markdown-it on Node (JIT and --jitless), on the same text: the CommonMark
// spec's examples, joined and repeated. Needs markdown-it: MARKDOWN_IT=<path to its folder>.
//
// node packages/markdown/scripts/bench.mjs commonmark.json 1,10,50
import { spawnSync } from "node:child_process"
import { readFileSync, writeFileSync } from "node:fs"
import { dirname, join } from "node:path"
import { fileURLToPath } from "node:url"
const root = join(dirname(fileURLToPath(import.meta.url)), "..")
const weft = process.env.WEFT_BIN ?? join(root, "../../core/target/release/weft")
const spec = process.argv[2]
const sizes = (process.argv[3] ?? "1,10,50").split(",").map(Number)
const lib = process.env.MARKDOWN_IT
const script = `
const MarkdownIt = require(${JSON.stringify(lib)})
const fs = require("fs")
const cases = JSON.parse(fs.readFileSync(${JSON.stringify(spec)}, "utf8"))
const text = cases.map((c) => c.markdown).join("\\n\\n").repeat(Number(process.argv[2]))
const md = new MarkdownIt({ html: false, linkify: true })
const start = process.hrtime.bigint()
const html = md.render(text)
console.log(Number(process.hrtime.bigint() - start) / 1e6, text.length, html.length)
`
const file = join(process.env.TMPDIR ?? "/workspace/core/target/tools", "bench-markdown-it.cjs")
writeFileSync(file, script)
function ms(cmd, args, env = {}) {
const start = process.hrtime.bigint()
const r = spawnSync(cmd, args, { cwd: root, encoding: "utf8", env: { ...process.env, ...env } })
if (r.status !== 0) throw new Error(r.stderr)
return { ms: Number(process.hrtime.bigint() - start) / 1e6, out: r.stdout.trim() }
}
const limits = { WEFT_FUEL: "100000000000", WEFT_MEMORY_MB: "4000" }
const base = Math.min(...[0, 1, 2].map(() => ms(weft, ["run", "bench/bench.weft", "0"], limits).ms))
console.log(`VM start-up (loading the package and the spec's text): ${base.toFixed(0)} ms`)
console.log("size KB | weft VM ms | markdown-it JIT ms | markdown-it --jitless ms | VM / JIT | VM / jitless")
for (const n of sizes) {
const vm = Math.min(...[0, 1, 2].map(() => ms(weft, ["run", "bench/bench.weft", String(n)], limits).ms)) - base
const best = (flags) => {
const runs = [0, 1, 2].map(() => ms("node", [...flags, file, String(n)]).out.split(" ").map(Number))
return { ms: Math.min(...runs.map((r) => r[0])), size: runs[0][1] }
}
const jit = best([])
const nojit = best(["--jitless"])
console.log(`${(jit.size / 1024).toFixed(0).padStart(7)} | ${vm.toFixed(0).padStart(10)} | ${jit.ms.toFixed(1).padStart(18)} | ${nojit.ms.toFixed(1).padStart(24)} | ${(vm / jit.ms).toFixed(1).padStart(8)} | ${(vm / nojit.ms).toFixed(1).padStart(12)}`)
}

View file

@ -0,0 +1,69 @@
// Generates tests/cases.weft from the CommonMark spec's examples and cmark-gfm's spec text.
//
// curl -o commonmark.json https://spec.commonmark.org/0.31.2/spec.json
// curl -o gfm.txt https://raw.githubusercontent.com/github/cmark-gfm/master/test/spec.txt
// node packages/markdown/scripts/gen-spec.mjs commonmark.json gfm.txt
import { mkdirSync, readFileSync, writeFileSync } from "node:fs"
import { dirname, join } from "node:path"
import { fileURLToPath } from "node:url"
const tests = join(dirname(fileURLToPath(import.meta.url)), "..", "tests")
mkdirSync(tests, { recursive: true })
const commonmark = JSON.parse(readFileSync(process.argv[2], "utf8")).map((c) => ({
id: c.example,
section: c.section,
markdown: c.markdown,
html: c.html,
}))
// cmark-gfm's spec.txt: examples are fenced by 32 backticks, `example` then the extension's name.
const gfm = []
let section = ""
let number = 0
const lines = readFileSync(process.argv[3], "utf8").split("\n")
for (let i = 0; i < lines.length; i++) {
const heading = lines[i].match(/^#{1,6} (.*)$/)
if (heading) section = heading[1]
const open = lines[i].match(/^`{32} example(.*)$/)
if (!open) continue
number++
const extensions = open[1].trim().split(/\s+/).filter(Boolean)
let j = i + 1
const source = []
while (lines[j] !== ".") source.push(lines[j++])
j++
const html = []
while (!lines[j].startsWith("`".repeat(32))) html.push(lines[j++])
i = j
// `disabled` examples describe behaviour cmark-gfm itself doesn't have. `tagfilter` is for pages that embed raw HTML:
// here `html: "escape"` and `"sanitize"` already remove those tags, and `"trust"` means what it says.
if (extensions.includes("disabled") || extensions.includes("tagfilter")) continue
const text = (l) => l.map((x) => x + "\n").join("").replaceAll("→", "\t")
gfm.push({ id: number, section, markdown: text(source), html: text(html), extensions })
}
const str = (s) => JSON.stringify(s).replaceAll("\u2028", "\\u2028").replaceAll("\u2029", "\\u2029").replaceAll("$", "\\$")
const row = (c) =>
` { id: ${c.id}, section: ${str(c.section)}, markdown: ${str(c.markdown)}, html: ${str(c.html)}, gfm: ${(c.extensions ?? []).length > 0} },`
writeFileSync(
join(tests, "cases.weft"),
`// Generated by scripts/gen-spec.mjs. Don't edit it.
/// One example. With gfm set it needs GFM's extensions; cmark-gfm runs the others without them.
export type Case = { id: number, section: string, markdown: string, html: string, gfm: boolean }
/// The examples of the CommonMark spec, version 0.31.2.
export let COMMONMARK: Case[] = [
${commonmark.map(row).join("\n")}
]
/// The examples of the GitHub Flavored Markdown spec.
export let GFM: Case[] = [
${gfm.map(row).join("\n")}
]
`,
)
console.log(`${commonmark.length} CommonMark examples, ${gfm.length} GFM examples`)

View file

@ -0,0 +1,73 @@
// Generates lib/entities.weft (the HTML5 named character references) and lib/unicode.weft (Unicode
// punctuation and symbols, what CommonMark's delimiter rules call punctuation).
//
// curl -o entities.json https://html.spec.whatwg.org/entities.json
// node packages/markdown/scripts/gen-tables.mjs entities.json
import { readFileSync, writeFileSync } from "node:fs"
import { dirname, join } from "node:path"
import { fileURLToPath } from "node:url"
const lib = join(dirname(fileURLToPath(import.meta.url)), "..", "lib")
const entities = JSON.parse(readFileSync(process.argv[2], "utf8"))
const pairs = Object.entries(entities)
.filter(([name]) => name.endsWith(";"))
.map(([name, v]) => `${name.slice(1, -1)}=${v.codepoints.map((c) => c.toString(16)).join(",")}`)
.sort()
writeFileSync(
join(lib, "entities.weft"),
`// Generated by scripts/gen-tables.mjs from https://html.spec.whatwg.org/entities.json. Don't edit it.
import { parseInt } from "std/math"
let DATA = "${pairs.join(" ")}"
fn build(): Map<string, string> {
let mut table = Map<string, string>()
for entry of DATA.split(" ") {
let at = entry.indexOf("=")
let mut text = ""
for hex of entry.slice(at + 1).split(",") {
text += String.fromCodePoint(try parseInt(hex, 16) else 0xfffd)
}
table.set(entry.slice(0, at), text)
}
return table
}
let ENTITIES = build()
/// The text of a named character reference (\`amp\` for \`&amp;\`), or \`null\` when HTML has no such name.
export fn entity(name: string): string? {
return ENTITIES.get(name)
}
`,
)
const punct = /[\p{P}\p{S}]/u
const ranges = []
let start = -1
for (let c = 0; c <= 0x110000; c++) {
const is = c < 0x110000 && !(c >= 0xd800 && c <= 0xdfff) && punct.test(String.fromCodePoint(c))
if (is && start < 0) start = c
if (!is && start >= 0) {
ranges.push(start, c - 1)
start = -1
}
}
const rows = []
for (let i = 0; i < ranges.length; i += 16) rows.push(" " + ranges.slice(i, i + 16).join(", ") + ",")
writeFileSync(
join(lib, "unicode.weft"),
`// Generated by scripts/gen-tables.mjs from the engine's Unicode tables (general categories P and S). Don't edit it.
/// Sorted, inclusive ranges of code points: start, end, start, end, …
export let PUNCTUATION: number[] = [
${rows.join("\n")}
]
`,
)
console.log(`${pairs.length} entities, ${ranges.length / 2} punctuation ranges`)

View file

@ -0,0 +1,39 @@
// Runs tests/hostile.weft: each input at n and 2n, timed from outside the VM (the clock doesn't move within a turn).
// A size that doubles must not take much more than twice as long.
//
// node packages/markdown/scripts/hostile.mjs [n]
import { spawnSync } from "node:child_process"
import { dirname, join } from "node:path"
import { fileURLToPath } from "node:url"
const root = join(dirname(fileURLToPath(import.meta.url)), "..")
const weft = process.env.WEFT_BIN ?? join(root, "../../core/target/release/weft")
const n = Number(process.argv[2] ?? 8000)
// The limits are lifted so that a slow path shows up as time rather than as a stop.
const env = { ...process.env, WEFT_FUEL: "20000000000", WEFT_MEMORY_MB: "1500" }
function run(args) {
const start = process.hrtime.bigint()
const r = spawnSync(weft, ["run", "tests/hostile.weft", ...args], { cwd: root, env, encoding: "utf8", timeout: 120000 })
return { ms: Number(process.hrtime.bigint() - start) / 1e6, out: r.stdout.trim(), err: r.stderr.trim(), status: r.status }
}
const names = run(["--list"]).out.split("\n")
const base = run(["lines", "0"]).ms
let bad = 0
for (const name of names) {
const a = run([name, String(n)])
const b = run([name, String(n * 2)])
const [ta, tb] = [Math.max(a.ms - base, 1), Math.max(b.ms - base, 1)]
const stopped = a.status !== 0 || b.status !== 0
const quadratic = tb > 1500 && tb / ta > 3.2
if (stopped || quadratic) bad++
const note = stopped ? ` STOPPED: ${(b.err || a.err).split("\n")[0]}` : quadratic ? " QUADRATIC" : ""
console.log(`${name.padEnd(24)} ${ta.toFixed(0).padStart(7)} ms ${tb.toFixed(0).padStart(7)} ms${note}`)
}
const expansion = run(["expansion"])
const size = Number(expansion.out)
if (!(size > 0 && size < 1000000)) bad++
console.log(`reference expansion: ${size} bytes of HTML from 50 KB of Markdown`)
console.log(bad === 0 ? "ok" : `${bad} failed`)
process.exit(bad === 0 ? 0 : 1)

1333
markdown/tests/cases.weft Normal file

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,47 @@
// Hostile input, one per run: `weft run tests/hostile.weft <name> <n>` renders it and prints the HTML's length.
// scripts/hostile.mjs runs them all and checks that the time each takes grows in proportion to its size.
import { log } from "std/log"
import { args } from "std/process"
import { toHtml } from "../markdown"
type Input = { name: string, make: (n: int) => string }
let inputs: Input[] = [
{ name: "nested quotes", make: n => ">".repeat(n) + " x" },
{ name: "nested lists", make: n => "- ".repeat(n) + "x" },
{ name: "nested emphasis", make: n => "*a **b ".repeat(n) + "c" + "** d*".repeat(n) },
{ name: "nested links", make: n => "[a ".repeat(n) + "](/x)".repeat(n) },
{ name: "nested images", make: n => "![".repeat(n) + "x" + "](/y)".repeat(n) },
{ name: "open emphasis", make: n => "*a ".repeat(n) },
{ name: "closed emphasis", make: n => "_a_ ".repeat(n) },
{ name: "open brackets", make: n => "[".repeat(n) },
{ name: "close brackets", make: n => "]".repeat(n) },
{ name: "unfinished links", make: n => "[a](".repeat(n) },
{ name: "backtick runs", make: n => "`a ``b ".repeat(n) },
{ name: "open tags", make: n => "<a ".repeat(n) },
{ name: "open comments", make: n => "a <!-- ".repeat(n) },
{ name: "open cdata", make: n => "a <![CDATA[ ".repeat(n) },
{ name: "open instructions", make: n => "a <? ".repeat(n) },
{ name: "entities", make: n => "&amp".repeat(n) },
{ name: "backslashes", make: n => "\\".repeat(n) },
{ name: "lines", make: n => "a\n".repeat(n) },
{ name: "reference definitions", make: n => "[a]: /b\n".repeat(n) },
{ name: "list items", make: n => "- a\n".repeat(n) },
{ name: "trailing parens", make: n => "http://a.dev/" + ")".repeat(n) },
{ name: "at signs", make: n => "a@".repeat(n) },
{ name: "table rows", make: n => "| a | b |\n|---|---|\n" + "| 1 | 2 |\n".repeat(n) },
{ name: "wide table", make: n => "| a ".repeat(n) + "\n" + "|-".repeat(n) },
]
let name = args().at(0) ?? "--list"
if name == "--list" {
log(inputs.map(i => i.name).join("\n"))
} else if name == "expansion" {
// A definition used many times can't multiply the output.
let url = "/" + "x".repeat(10000)
log(toHtml(`[a]: ${url}\n\n` + "[a] ".repeat(10000)).length)
} else {
let n = try int(args().at(1) ?? "1000") else 1000
let input = inputs.find(i => i.name == name)
log(input == null ? -1 : toHtml(input.make(n)).length)
}

View file

@ -0,0 +1,220 @@
// `weft test`: the package's API and its safety. The spec's examples are in tests/spec.weft, hostile input in
// tests/hostile.weft (both are programs: they run past a test's deadline).
import { assert, assertEq } from "std/testing"
import { parse, render, sanitize, toHtml, toText, visit } from "../markdown"
// ---- safe by default ----
test "raw HTML is shown as text" {
assertEq(toHtml("<script>alert(1)</script>"), "<p>&lt;script&gt;alert(1)&lt;/script&gt;</p>\n")
assertEq(
toHtml("a <b onclick=\"x()\">b</b>"),
"<p>a &lt;b onclick=&quot;x()&quot;&gt;b&lt;/b&gt;</p>\n",
)
}
test "unsafe link schemes render as their text" {
let attacks = [
"[x](javascript:alert(1))",
"[x](JaVaScRiPt:alert(1))",
"[x](vbscript:msgbox(1))",
"[x](data:text/html;base64,PHNjcmlwdD4=)",
"[x](&#106;avascript:alert(1))",
"[x](javascript&colon;alert(1))",
"[x](file:///etc/passwd)",
]
for a of attacks {
assertEq(toHtml(a), "<p>x</p>\n")
}
assertEq(toHtml("<javascript:alert(1)>"), "<p>javascript:alert(1)</p>\n")
}
test "unsafe images render as their alt text" {
assertEq(toHtml("![cat](javascript:alert(1))"), "<p>cat</p>\n")
assertEq(toHtml("![cat](data:image/svg+xml;base64,PHN2Zz4=)"), "<p>cat</p>\n")
assertEq(
toHtml("![cat](data:image/png;base64,AAAA)"),
"<p><img src=\"data:image/png;base64,AAAA\" alt=\"cat\" /></p>\n",
)
}
test "safe URLs are kept" {
assertEq(
toHtml("[a](https://x.dev/?q=1&r=2)"),
"<p><a href=\"https://x.dev/?q=1&amp;r=2\">a</a></p>\n",
)
assertEq(toHtml("[a](/docs/x:y)"), "<p><a href=\"/docs/x:y\">a</a></p>\n")
assertEq(toHtml("[a](#top)"), "<p><a href=\"#top\">a</a></p>\n")
assertEq(toHtml("[a](mailto:me@x.dev)"), "<p><a href=\"mailto:me@x.dev\">a</a></p>\n")
assertEq(toHtml("[a](tel:+38640123456)"), "<p><a href=\"tel:+38640123456\">a</a></p>\n")
}
test "titles, code and attributes are escaped" {
assertEq(
toHtml("[a](/x \"\\\"><script>\")"),
"<p><a href=\"/x\" title=\"&quot;&gt;&lt;script&gt;\">a</a></p>\n",
)
assertEq(toHtml("`<b>`"), "<p><code>&lt;b&gt;</code></p>\n")
assertEq(
toHtml("```\"><script>\nx\n```"),
"<pre><code class=\"language-&quot;&gt;&lt;script&gt;\">x\n</code></pre>\n",
)
}
// ---- the sanitiser ----
test "sanitize removes scripts, handlers and unsafe URLs" {
assertEq(sanitize("<p onclick=\"x()\">hi<script>bad()</script></p>"), "<p>hi</p>")
assertEq(sanitize("<a href=\"javascript:x()\" title=\"t\">hi</a>"), "<a title=\"t\">hi</a>")
assertEq(sanitize("<a href=\"java&#115;cript:x()\">hi</a>"), "<a>hi</a>")
assertEq(sanitize("<a href=\"java&#9;script:x()\">hi</a>"), "<a>hi</a>")
assertEq(sanitize("<a href=\" &#14; javascript:x()\">hi</a>"), "<a>hi</a>")
assertEq(sanitize("<img src=x onerror=alert(1)>"), "<img src=\"x\" />")
assertEq(sanitize("<IMG SRC=\"JAVASCRIPT:x\">"), "<img />")
assertEq(sanitize("<svg><script>x</script></svg>ok"), "ok")
assertEq(sanitize("<math><mi>x</mi></math>ok"), "ok")
assertEq(sanitize("<style>*{display:none}</style>ok"), "ok")
assertEq(sanitize("<iframe src=\"https://evil\"></iframe>ok"), "ok")
assertEq(sanitize("<ScRiPt >x</sCrIpT >ok"), "ok")
assertEq(sanitize("<script>never closed"), "")
assertEq(sanitize("<!-- <script>x</script> -->ok"), "ok")
assertEq(sanitize("<form action=\"/x\"><button>go</button></form>"), "go")
}
test "sanitize keeps only a disabled checkbox" {
assertEq(sanitize("<input type=\"text\" value=\"pw\">"), "")
assertEq(sanitize("<input>"), "")
assertEq(
sanitize("<input type=\"checkbox\" checked>"),
"<input type=\"checkbox\" checked=\"\" disabled=\"\" />",
)
}
test "sanitize gives well-formed output" {
assertEq(sanitize("<b><i>x</b>y"), "<b><i>x</i></b>y")
assertEq(sanitize("</div>x"), "x")
assertEq(sanitize("<p>a"), "<p>a</p>")
assertEq(sanitize("1 < 2 & 3 > 2 &amp; ok"), "1 &lt; 2 &amp; 3 &gt; 2 &amp; ok")
assertEq(sanitize("<a title=\"a&quot;b\">x</a>"), "<a title=\"a&quot;b\">x</a>")
assertEq(
sanitize("<a target=\"_blank\" href=\"/x\">x</a>"),
"<a target=\"_blank\" href=\"/x\" rel=\"noopener noreferrer\">x</a>",
)
assertEq(sanitize("<a target=\"evil\" href=\"/x\">x</a>"), "<a href=\"/x\">x</a>")
}
test "a policy narrows what is kept" {
assertEq(sanitize("<b>x</b><i>y</i>", { tags: ["b"] }), "<b>x</b>y")
assertEq(sanitize("<a href=\"ftp://x\">y</a>", { schemes: ["ftp"] }), "<a href=\"ftp://x\">y</a>")
assertEq(sanitize("<a href=\"https://x\">y</a>", { schemes: ["ftp"] }), "<a>y</a>")
let attributes = Map<string, string[]>([["b", ["title"]]])
assertEq(
sanitize("<b title=\"t\" id=\"i\">x</b>", { tags: ["b"], attributes }),
"<b title=\"t\">x</b>",
)
}
test "html: sanitize keeps safe raw HTML in Markdown" {
assertEq(
toHtml("<div onclick=\"x()\">\n*hi*\n</div>", { html: "sanitize" }),
"<div>\n*hi*\n</div>\n",
)
assertEq(
toHtml("a <kbd>b</kbd> <script>c</script>", { html: "sanitize" }),
"<p>a <kbd>b</kbd> </p>\n",
)
assertEq(toHtml("<script>\nbad()\n\n</script>\n\nok", { html: "sanitize" }), "\n<p>ok</p>\n")
assertEq(
toHtml("- [x] done", { html: "sanitize" }),
"<ul>\n<li><input checked=\"\" disabled=\"\" type=\"checkbox\" /> done</li>\n</ul>\n",
)
}
test "html: trust passes raw HTML through" {
assertEq(
toHtml("<div onclick=\"x()\">a</div>", { html: "trust" }),
"<div onclick=\"x()\">a</div>\n",
)
}
// ---- GFM ----
test "tables" {
let html = toHtml("| a | b |\n|:--|--:|\n| 1 | `x\\|y` |")
assertEq(
html,
"<table>\n<thead>\n<tr>\n<th align=\"left\">a</th>\n<th align=\"right\">b</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td align=\"left\">1</td>\n<td align=\"right\"><code>x|y</code></td>\n</tr>\n</tbody>\n</table>\n",
)
}
test "task lists" {
let doc = parse("- [ ] todo\n- [x] done\n- plain")
match doc.children[0] {
BulletList { items } => {
let checked: boolean?[] = items.map(i => i.checked)
let expected: boolean?[] = [false, true, null]
assertEq(checked, expected)
},
_ => assert(false, "a list"),
}
}
test "strikethrough and bare links" {
assertEq(toHtml("~~gone~~"), "<p><del>gone</del></p>\n")
assertEq(
toHtml("see https://x.dev/a_b."),
"<p>see <a href=\"https://x.dev/a_b\">https://x.dev/a_b</a>.</p>\n",
)
assertEq(toHtml("www.x.dev"), "<p><a href=\"http://www.x.dev\">www.x.dev</a></p>\n")
assertEq(toHtml("me@x.dev"), "<p><a href=\"mailto:me@x.dev\">me@x.dev</a></p>\n")
}
test "gfm: false is plain CommonMark" {
assertEq(toHtml("~~a~~ https://x.dev", { gfm: false }), "<p>~~a~~ https://x.dev</p>\n")
assertEq(toHtml("| a |\n|---|", { gfm: false }), "<p>| a |\n|---|</p>\n")
}
// ---- what Vidni's briefs need ----
test "heading offset, link target and URL rewriting" {
assertEq(toHtml("# T", { headingOffset: 1 }), "<h2>T</h2>\n")
assertEq(toHtml("###### T", { headingOffset: 3 }), "<h6>T</h6>\n")
assertEq(
toHtml("[a](https://x.dev)", { linkTarget: "_blank" }),
"<p><a href=\"https://x.dev\" target=\"_blank\" rel=\"noopener noreferrer\">a</a></p>\n",
)
let resolve = (url: string, kind: "link" | "image") => url.startsWith("upload:")
? `https://cdn.x.dev/${url.slice(7)}`
: null
assertEq(
toHtml("![c](upload:abc)", { rewriteUrl: resolve }),
"<p><img src=\"https://cdn.x.dev/abc\" alt=\"c\" /></p>\n",
)
// Not resolved: `upload:` isn't an allowed scheme, so the caption stays.
assertEq(toHtml("![c](upload:abc)"), "<p>c</p>\n")
}
test "toText for previews and emails" {
assertEq(
toText(parse("# Hi\n\nRead **this** [guide](https://x.dev).\n\n![cat](a.png)\n\n- one\n- two")),
"Hi\n\nRead this guide (https://x.dev).\n\n[image: cat]\n\n- one\n\n- two",
)
}
test "visit finds the images of a document" {
let mut urls: string[] = []
visit(parse("![a](upload:1)\n\n> *![b](upload:2)*\n\n| ![c](upload:3) |\n|---|"), n => {
if let Picture { url } = n {
urls.push(url)
}
})
assertEq(urls, ["upload:1", "upload:2", "upload:3"])
}
test "render uses a tree that was changed" {
let doc = parse("hello")
assertEq(render({ children: [...doc.children, ...doc.children] }), "<p>hello</p>\n<p>hello</p>\n")
}
test "strange characters survive" {
assertEq(toHtml("a\u{0}b"), "<p>a\u{fffd}b</p>\n")
assertEq(toHtml("line\r\nnext\rlast"), "<p>line\nnext\nlast</p>\n")
assertEq(toHtml("😀 *é*"), "<p>😀 <em>é</em></p>\n")
assertEq(toHtml(""), "")
}

54
markdown/tests/spec.weft Normal file
View file

@ -0,0 +1,54 @@
// Runs the spec's examples: `weft run tests/spec.weft [commonmark|gfm] [-v]` prints every failure and a summary.
import { args, exit } from "std/process"
import { log } from "std/log"
import { toHtml } from "../markdown"
import { COMMONMARK, GFM } from "./cases"
import type { Case } from "./cases"
/// Where the GFM spec (written against CommonMark 0.29) and CommonMark 0.31.2 give different HTML for the same text,
/// CommonMark wins: cmark-gfm's renderer folds nested `<strong>`, which CommonMark's examples keep.
fn followsCommonMark(c: Case, got: string): boolean {
return COMMONMARK.some(o => o.markdown == c.markdown && o.html == got)
}
fn runAll(name: string, cases: Case[], verbose: boolean): int {
let mut failed = 0
let mut disagreements = 0
let mut bySection = Map<string, int>()
for c of cases {
let got = toHtml(c.markdown, { gfm: c.gfm, html: "trust" })
if got != c.html && followsCommonMark(c, got) {
disagreements += 1
} else if got != c.html {
failed += 1
bySection.set(c.section, (bySection.get(c.section) ?? 0) + 1)
if verbose {
log(`--- ${name} example ${c.id} (${c.section})`)
log(`markdown: ${c.markdown}`)
log(`expected: ${c.html}`)
log(`got: ${got}`)
}
}
}
log(
`${name}: ${cases.length - failed - disagreements} of ${cases.length} pass, ${disagreements} follow CommonMark 0.31.2 where the specs disagree, ${failed} fail`,
)
for [section, n] of bySection {
log(` ${n} failing in ${section}`)
}
return failed
}
let argv = args()
let verbose = argv.includes("-v")
let only = argv.filter(a => a != "-v").at(0)
let mut failures = 0
if only == null || only == "commonmark" {
failures += runAll("CommonMark", COMMONMARK, verbose)
}
if only == null || only == "gfm" {
failures += runAll("GFM", GFM, verbose)
}
if failures > 0 {
exit(1)
}

10
markdown/weft.toml Normal file
View file

@ -0,0 +1,10 @@
[package]
name = "weft/markdown"
version = "0.1.0"
description = "CommonMark and GitHub Flavored Markdown: an AST, HTML and a sanitiser, safe by default"
license = "MIT OR Apache-2.0"
# A placeholder until the public repository exists (packages/README.md, Publishing for real).
repository = "https://git.weft-lang.dev/weft/packages"
exports = ["markdown", "ast"]
main = "markdown"
include = ["markdown.weft", "ast.weft", "lib", "README.md"]

143
scripts/verify-local.mjs Normal file
View file

@ -0,0 +1,143 @@
// Proves the official packages end to end against the registry app running locally: starts `apps/registry`, makes a
// publish token and a signing key for the owner `weft`, publishes each package signed, then in a fresh app runs
// `weft add`, checks it, runs it on the VM and builds it for the browser. Nothing leaves this machine.
//
// node packages/scripts/verify-local.mjs [markdown] [stripe] [email]
import { spawn, spawnSync } from "node:child_process"
import { existsSync, mkdirSync, rmSync, writeFileSync } from "node:fs"
import { dirname, join } from "node:path"
import { fileURLToPath } from "node:url"
const repo = join(dirname(fileURLToPath(import.meta.url)), "../..")
const weft = process.env.WEFT_BIN ?? join(repo, "core/target/release/weft")
const work = join(repo, "core/target/tools/verify")
const admin = "local-operator-token"
const port = 4873 + (process.pid % 1000)
const registry = `http://127.0.0.1:${port}`
// What a fresh app does with each package, and the line it must print. No network: these run offline.
const apps = {
markdown: {
main: `import { toHtml } from "@weft/markdown"
import { log } from "std/log"
log(toHtml("# Hello\\n\\n- [x] **done** <script>bad()</script>"))
`,
expect: "<h1>Hello</h1>\n<ul>\n<li><input checked=\"\" disabled=\"\" type=\"checkbox\"> <strong>done</strong> &lt;script&gt;bad()&lt;/script&gt;</li>\n</ul>",
capabilities: "capabilities none",
},
stripe: {
main: `import { client } from "@weft/stripe"
import { signature, verify } from "@weft/stripe/webhooks"
import { log } from "std/log"
let payload = "{\\"id\\":\\"evt_1\\",\\"type\\":\\"checkout.session.completed\\",\\"created\\":1,\\"data\\":{\\"object\\":{}}}"
let event = try verify(payload, signature(payload, "whsec_local", 1700000000), "whsec_local", { now: 1700000000 })
log(event.type, client("sk_test_local").baseUrl)
`,
expect: "checkout.session.completed https://api.stripe.com",
capabilities: "capabilities http.fetch",
},
email: {
main: `import { memory, outbox, send } from "@weft/email"
import { toMime } from "@weft/email/mime"
import { document } from "@weft/email/template"
import { render } from "std/app"
import { log } from "std/log"
import Welcome from "./Welcome"
let body = document(render(Welcome), { title: "Hi" })
let sent = try await send(memory(), { from: "a@example.com", to: ["b@example.com"], subject: "Hi", html: body.html, text: body.text })
log(sent.provider, outbox().length, toMime(outbox()[0], { id: "x" }).includes("multipart/alternative"))
`,
files: {
"Welcome.weft": `import Button from "@weft/email/components/Button"
import H1 from "@weft/email/components/H1"
import P from "@weft/email/components/P"
---
<H1>Hi</H1>
<P>Welcome</P>
<Button url="https://example.com" label="Open" />
`,
},
expect: "memory 1 true",
capabilities: "capabilities http.fetch",
},
}
rmSync(work, { recursive: true, force: true })
mkdirSync(work, { recursive: true })
const base = { ...process.env, WEFT_REGISTRY: registry, WEFT_CACHE: join(work, "cache") }
delete base.WEFT_TOKEN
delete base.WEFT_SIGNING_KEY
function run(args, cwd, env = {}) {
const r = spawnSync(weft, args, { cwd, env: { ...base, ...env }, encoding: "utf8" })
const out = `${r.stdout}${r.stderr}`
if (r.status !== 0) throw new Error(`weft ${args.join(" ")} failed in ${cwd}:\n${out}`)
return out
}
async function api(method, path, token, body) {
const res = await fetch(`${registry}${path}`, {
method,
headers: { authorization: `Bearer ${token}`, "content-type": "application/json" },
body: JSON.stringify(body),
})
const text = await res.text()
if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${text}`)
return JSON.parse(text)
}
const server = spawn(weft, ["run", "."], {
cwd: join(repo, "apps/registry"),
env: { ...base, PORT: String(port), HOST: "127.0.0.1", REGISTRY_DB: join(work, "registry.db"), REGISTRY_ADMIN_TOKEN: admin },
stdio: ["ignore", "ignore", "inherit"],
})
try {
for (let i = 0; ; i++) {
const up = await fetch(`${registry}/config.json`).then((r) => r.ok, () => false)
if (up) break
if (i > 600) throw new Error("the registry didn't start")
await new Promise((r) => setTimeout(r, 100))
}
const { token } = await api("POST", "/v1/tokens", admin, { owners: ["weft"], label: "official packages" })
const keys = run(["publish", "--new-key"], work)
const field = (label) => keys.split("\n").find((l) => l.startsWith(label)).split(" ").pop()
const [secret, key] = [field("secret"), field("public")]
// The operator's request, from this machine: a first key sent with an owner's token waits a day.
const proof = run(["publish", "--endorse-key", key, "--owner", "weft"], work, { WEFT_SIGNING_KEY: secret }).trim()
await api("PUT", "/v1/owners/weft/key", admin, { key, proof })
console.log(`registry ${registry}: owner \`weft\` holds signing key ${key.slice(0, 16)}…`)
for (const name of process.argv.slice(2).length ? process.argv.slice(2) : Object.keys(apps)) {
const dir = join(repo, "packages", name)
const signing = { WEFT_TOKEN: token, WEFT_SIGNING_KEY: secret, WEFT_COMMIT: "local" }
console.log(`\n== weft/${name}`)
console.log(run(["publish", "--dry-run"], dir).trim().split("\n").filter((l) => /capabilit|dry run/.test(l)).join("\n"))
console.log(run(["publish", "--sign"], dir, signing).trim())
// Without the key the registry refuses a new version: signing is what's protecting the owner.
const app = join(work, `app-${name}`)
mkdirSync(app, { recursive: true })
writeFileSync(join(app, "weft.toml"), 'main = "main.weft"\n\n[dependencies]\n')
writeFileSync(join(app, "main.weft"), apps[name].main)
for (const [file, source] of Object.entries(apps[name].files ?? {})) writeFileSync(join(app, file), source)
const added = run(["add", `weft/${name}`], app)
console.log(added.trim())
if (!added.includes(apps[name].capabilities)) throw new Error(`expected "${apps[name].capabilities}" in: ${added}`)
run(["check", "."], app)
const printed = run(["run", "."], app).trim()
if (printed !== apps[name].expect) throw new Error(`weft run printed:\n${printed}\nexpected:\n${apps[name].expect}`)
console.log("weft run: the expected output")
run(["build", "--target", "js", "."], app)
if (!existsSync(join(app, "build"))) throw new Error("weft build wrote nothing")
console.log("weft build --target js: ok")
run(["install", "--frozen"], app)
console.log("weft install --frozen: ok")
const page = await fetch(`${registry}/packages/weft/${name}`).then((r) => r.text())
if (!page.includes(`weft/${name}`)) throw new Error("the registry has no page for it")
console.log(`generated docs: ${registry}/packages/weft/${name} (${page.length} bytes)`)
}
console.log("\nverified")
} finally {
server.kill()
}

148
stripe/README.md Normal file
View file

@ -0,0 +1,148 @@
# @weft/stripe
A typed client for Stripe's API, written in Weft over `std/http`. It covers customers, Checkout, payment intents,
subscriptions, prices, products, refunds, the customer portal and webhooks. Its only capability is `http.fetch@api.stripe.com`.
```sh
weft add weft/stripe
```
```weft
import { client } from "@weft/stripe"
import * as checkout from "@weft/stripe/checkout"
import { env } from "std/process"
let stripe = client(env("STRIPE_SECRET_KEY")!)
let session = try await checkout.create(stripe, {
mode: "payment",
success_url: "https://example.com/thanks?s={CHECKOUT_SESSION_ID}",
cancel_url: "https://example.com/cart",
line_items: [{ quantity: 1, price_data: { currency: "eur", unit_amount: 1999, product_data: { name: "Ticket" } } }],
}, { idempotencyKey: `sale-${sale.id}` })
```
## Modules
Each resource is its own module, imported as `@weft/stripe/<name>`:
| module | functions |
|---|---|
| `customers` | `create`, `retrieve`, `update`, `remove`, `list`, `listAll`, `search` |
| `checkout` | `create`, `retrieve`, `expire`, `list`, `listAll`, `lineItems` |
| `payment_intents` | `create`, `retrieve`, `update`, `confirm`, `capture`, `cancel`, `list`, `listAll`, `search` |
| `subscriptions` | `create`, `retrieve`, `update`, `cancel`, `list`, `listAll`, `search` |
| `prices` | `create`, `retrieve`, `update`, `list`, `listAll` |
| `products` | `create`, `retrieve`, `update`, `remove`, `list`, `listAll` |
| `refunds` | `create`, `retrieve`, `list`, `listAll` |
| `billing` | `createPortalSession`, `retrieveCoupon`, `retrievePromotionCode`, `createAccountSession` |
| `webhooks` | `verify`, `signature` |
Field names are Stripe's own, so its API reference applies as written. Every parameter type has an `extra` field for what the
type doesn't list, and `request` calls anything else:
```weft
import { request } from "@weft/stripe"
type Balance = { livemode: boolean }
let balance = try await request(stripe, "GET", "/v1/balance", null, {}, j => try j as Balance)
```
## Errors are values
Every call fails with a `StripeError`:
```weft
let intent, err = try await paymentIntents.create(stripe, params)
if err != null {
return match err {
CardError { failure } => declined(failure.decline_code),
RateLimited | ConnectionFailed => retryLater(),
_ => report(errorMessage(err)),
}
}
```
| variant | when |
|---|---|
| `CardError` | the card was declined |
| `InvalidRequest` | the parameters are wrong, or the object doesn't exist |
| `AuthenticationFailed`, `PermissionDenied` | the API key is wrong or can't do this |
| `RateLimited` | too many requests |
| `IdempotencyConflict` | an idempotency key was reused with other parameters |
| `ApiFailure` | Stripe had a problem |
| `ConnectionFailed` | Stripe wasn't reached, so the outcome of a `POST` is unknown |
| `BadResponse` | Stripe's answer isn't what the types say |
## Retries and idempotency
A request is repeated up to `maxRetries` times (2) after a network error, a conflict or a server error, with exponential
backoff, as Stripe's own libraries do. It obeys Stripe's `Stripe-Should-Retry` and `Retry-After` headers. Every attempt of a
`POST` carries one idempotency key, so a request Stripe did receive isn't carried out twice. Pass `idempotencyKey` to make the
key your own: then a repeat after a crash is safe too.
## Paging
`list` gives one page, and `listAll` follows the pages up to a limit (10,000 items unless you pass `max`):
```weft
let page = try await customers.list(stripe, { limit: 20, starting_after: lastId })
let everyone = try await customers.listAll(stripe, { email: "ana@example.com" })
```
## Connect
Give `stripeAccount` to the client, or to one call, to act on a connected account:
```weft
let session = try await checkout.create(stripe, params, { stripeAccount: "acct_123" })
```
## Expandable fields and metadata
An expandable field is `json`: an id, or the object when you asked for it with `expand`. `idOf` reads either, and `as` gives
the object its type. `metadata` reads one value of an object's metadata.
```weft
let session = try await checkout.retrieve(stripe, id, { expand: ["subscription"] })
let subscriptionId = idOf(session.subscription)
let sub = try session.subscription as Subscription
let siteId = metadata(sub.metadata, "siteId")
```
## Webhooks
`verify` checks the signature in constant time and refuses events older than five minutes. Give it the body exactly as it
arrived:
```weft
import { verify } from "@weft/stripe/webhooks"
async fn webhook(req: Request) {
let event, err = try verify(req.body, req.headers.get("stripe-signature") ?? "", secret)
if err != null {
return status(400)
}
if event.type == "checkout.session.completed" {
let session = try event.data.object as Session else null
}
return json({ received: true })
}
```
`signature(payload, secret, timestamp)` makes the header for a test of your own handler.
## What has and hasn't been tested
`node packages/stripe/scripts/integration.mjs` runs every function above against [stripe-mock](https://github.com/stripe/stripe-mock), which
checks each request against Stripe's OpenAPI specification and answers from its fixtures, and runs retries, headers, paging and
errors against a scripted server. `weft test tests/stripe.test.weft` covers the signature code with FIPS 180-4's and RFC 4231's
vectors and a signature made by Node's `crypto`. **Nothing here has run against Stripe itself**: try a test-mode key before
relying on it.
## Limits
- **It reaches only `api.stripe.com`.** As a dependency, a `baseUrl` on any other host fails with a `CapabilityError`. To point
it at stripe-mock or a proxy, the app lists that host too: `allow = ["http.fetch@api.stripe.com", "http.fetch@127.0.0.1"]`.
- `metadata` and expandable fields are `json`: Weft's `as` can't turn a JSON object into a `Map`.
- Verifying a signature takes about 0.5 ms per KB of event. It is written in Weft until `std/crypto` is available.

126
stripe/billing.weft Normal file
View file

@ -0,0 +1,126 @@
// The customer portal, discounts and embedded Connect components.
import { request, segment } from "./stripe"
import type { Client, RequestOptions, StripeError } from "./stripe"
/// A session of the customer portal: send the customer to its `url`.
export type PortalSession = {
id: string,
url: string,
customer?: string,
return_url?: string,
created?: float,
}
/// A new portal session.
export type PortalParams = {
customer: string,
return_url?: string,
configuration?: string,
locale?: string,
extra?: json,
}
/// A coupon.
export type Coupon = {
id: string,
name?: string,
valid?: boolean,
percent_off?: float,
amount_off?: float,
currency?: string,
duration?: string,
duration_in_months?: float,
max_redemptions?: float,
times_redeemed?: float,
redeem_by?: float,
metadata?: json,
}
/// A promotion code customers can type in. `coupon` and `customer` are expandable: `idOf` gives their ids.
export type PromotionCode = {
id: string,
code: string,
active?: boolean,
coupon?: json,
promotion?: json,
customer?: json,
expires_at?: float,
max_redemptions?: float,
times_redeemed?: float,
metadata?: json,
}
/// A session for Stripe's embedded Connect components: give its `client_secret` to the browser.
export type AccountSession = {
account: string,
client_secret: string,
expires_at?: float,
livemode?: boolean,
}
/// A new account session. `components` says which embedded components it may show.
export type AccountSessionParams = { account: string, components: json }
/// Opens the customer portal for a customer.
export async fn createPortalSession(
c: Client,
params: PortalParams,
options: RequestOptions = {},
): PortalSession ! StripeError {
return try await request(
c,
"POST",
"/v1/billing_portal/sessions",
params,
options,
j => try j as PortalSession,
)
}
/// Gets a coupon by id.
export async fn retrieveCoupon(
c: Client,
id: string,
options: RequestOptions = {},
): Coupon ! StripeError {
return try await request(
c,
"GET",
`/v1/coupons/${segment(id)}`,
null,
options,
j => try j as Coupon,
)
}
/// Gets a promotion code by id.
export async fn retrievePromotionCode(
c: Client,
id: string,
options: RequestOptions = {},
): PromotionCode ! StripeError {
return try await request(
c,
"GET",
`/v1/promotion_codes/${segment(id)}`,
null,
options,
j => try j as PromotionCode,
)
}
/// Creates a session for a connected account's embedded components.
export async fn createAccountSession(
c: Client,
params: AccountSessionParams,
options: RequestOptions = {},
): AccountSession ! StripeError {
return try await request(
c,
"POST",
"/v1/account_sessions",
params,
options,
j => try j as AccountSession,
)
}

213
stripe/checkout.weft Normal file
View file

@ -0,0 +1,213 @@
// Checkout: Stripe's hosted payment page.
import { collect, request, segment } from "./stripe"
import type { Client, Page, RequestOptions, StripeError } from "./stripe"
/// A Checkout session. `customer`, `subscription` and `payment_intent` are expandable: `idOf` gives their ids.
export type Session = {
id: string,
url?: string,
status?: string,
payment_status?: string,
mode?: string,
amount_subtotal?: float,
amount_total?: float,
currency?: string,
client_reference_id?: string,
client_secret?: string,
customer?: json,
customer_email?: string,
customer_details?: json,
invoice?: json,
line_items?: json,
locale?: string,
metadata?: json,
payment_intent?: json,
subscription?: json,
success_url?: string,
cancel_url?: string,
expires_at?: float,
total_details?: json,
created?: float,
livemode?: boolean,
}
/// What a line is sold for, when it isn't an existing price.
export type PriceData = {
currency: string,
unit_amount?: float,
unit_amount_decimal?: string,
product?: string,
product_data?: {
name: string,
description?: string,
images?: string[],
metadata?: Map<string, string>,
},
recurring?: { interval: string, interval_count?: float },
tax_behavior?: string,
}
/// One line of what is being bought: an existing `price`, or `price_data`.
export type LineItem = { price?: string, price_data?: PriceData, quantity?: float, extra?: json }
/// A new Checkout session. `extra` holds any field this type doesn't list.
export type SessionParams = {
mode: "payment" | "subscription" | "setup",
success_url?: string,
cancel_url?: string,
return_url?: string,
ui_mode?: string,
line_items?: LineItem[],
customer?: string,
customer_email?: string,
customer_creation?: string,
client_reference_id?: string,
locale?: string,
metadata?: Map<string, string>,
expires_at?: float,
allow_promotion_codes?: boolean,
discounts?: { coupon?: string, promotion_code?: string }[],
payment_method_types?: string[],
billing_address_collection?: string,
automatic_tax?: { enabled: boolean },
payment_intent_data?: {
application_fee_amount?: float,
description?: string,
metadata?: Map<string, string>,
capture_method?: string,
receipt_email?: string,
transfer_data?: { destination: string, amount?: float },
extra?: json,
},
subscription_data?: {
application_fee_percent?: float,
description?: string,
metadata?: Map<string, string>,
trial_end?: float,
trial_period_days?: float,
extra?: json,
},
extra?: json,
}
/// What to list.
export type SessionListParams = {
customer?: string,
payment_intent?: string,
subscription?: string,
status?: string,
limit?: float,
starting_after?: string,
ending_before?: string,
extra?: json,
}
/// A line of a finished session.
export type SessionLineItem = {
id: string,
amount_subtotal?: float,
amount_total?: float,
currency?: string,
description?: string,
price?: json,
quantity?: float,
}
/// Creates a session: send the customer to its `url`.
///
/// ```weft
/// let session = try await checkout.create(stripe, {
/// mode: "payment",
/// success_url: "https://example.com/thanks?s={CHECKOUT_SESSION_ID}",
/// cancel_url: "https://example.com/cart",
/// line_items: [{ quantity: 1, price_data: { currency: "eur", unit_amount: 1999, product_data: { name: "Ticket" } } }],
/// }, { idempotencyKey: `sale-${sale.id}`, stripeAccount: accountId })
/// ```
export async fn create(
c: Client,
params: SessionParams,
options: RequestOptions = {},
): Session ! StripeError {
return try await request(
c,
"POST",
"/v1/checkout/sessions",
params,
options,
j => try j as Session,
)
}
/// Gets a session by id. `{ expand: ["subscription"] }` includes what it refers to.
export async fn retrieve(
c: Client,
id: string,
options: RequestOptions = {},
): Session ! StripeError {
return try await request(
c,
"GET",
`/v1/checkout/sessions/${segment(id)}`,
null,
options,
j => try j as Session,
)
}
/// Ends a session that hasn't been paid, so it can't be.
export async fn expire(c: Client, id: string, options: RequestOptions = {}): Session ! StripeError {
return try await request(
c,
"POST",
`/v1/checkout/sessions/${segment(id)}/expire`,
null,
options,
j => try j as Session,
)
}
/// One page of sessions, newest first.
export async fn list(c: Client, params: SessionListParams = {}, options: RequestOptions = {}): Page<
Session,
> ! StripeError {
return try await request(
c,
"GET",
"/v1/checkout/sessions",
params,
options,
j => try j as Page<Session>,
)
}
/// Every session matching `params`, up to `max`.
export async fn listAll(
c: Client,
params: SessionListParams = {},
options: RequestOptions = {},
max: int = 10000,
): Session[] ! StripeError {
return try await collect(
c,
"/v1/checkout/sessions",
params,
options,
j => try j as Page<Session>,
x => x.id,
max,
)
}
/// One page of what a session sold.
export async fn lineItems(c: Client, id: string, options: RequestOptions = {}): Page<
SessionLineItem,
> ! StripeError {
return try await request(
c,
"GET",
`/v1/checkout/sessions/${segment(id)}/line_items`,
null,
options,
j => try j as Page<SessionLineItem>,
)
}

150
stripe/customers.weft Normal file
View file

@ -0,0 +1,150 @@
// Customers.
import { collect, request, segment } from "./stripe"
import type {
Address,
Client,
Deleted,
Page,
RequestOptions,
SearchPage,
StripeError,
} from "./stripe"
/// A customer.
export type Customer = {
id: string,
email?: string,
name?: string,
phone?: string,
description?: string,
address?: Address,
metadata?: json,
balance?: float,
currency?: string,
delinquent?: boolean,
default_source?: json,
invoice_prefix?: string,
created?: float,
livemode?: boolean,
}
/// The fields of a customer you can set. `extra` holds any field this type doesn't list.
export type CustomerParams = {
email?: string,
name?: string,
phone?: string,
description?: string,
address?: Address,
metadata?: Map<string, string>,
payment_method?: string,
preferred_locales?: string[],
extra?: json,
}
/// What to list. `email` is matched exactly.
export type CustomerListParams = {
email?: string,
limit?: float,
starting_after?: string,
ending_before?: string,
extra?: json,
}
/// Creates a customer.
///
/// ```weft
/// let customer = try await customers.create(stripe, { email: "ana@example.com", name: "Ana" })
/// ```
export async fn create(
c: Client,
params: CustomerParams,
options: RequestOptions = {},
): Customer ! StripeError {
return try await request(c, "POST", "/v1/customers", params, options, j => try j as Customer)
}
/// Gets a customer by id.
export async fn retrieve(
c: Client,
id: string,
options: RequestOptions = {},
): Customer ! StripeError {
return try await request(
c,
"GET",
`/v1/customers/${segment(id)}`,
null,
options,
j => try j as Customer,
)
}
/// Changes the fields given.
export async fn update(
c: Client,
id: string,
params: CustomerParams,
options: RequestOptions = {},
): Customer ! StripeError {
return try await request(
c,
"POST",
`/v1/customers/${segment(id)}`,
params,
options,
j => try j as Customer,
)
}
/// Deletes a customer, and cancels their subscriptions.
export async fn remove(c: Client, id: string, options: RequestOptions = {}): Deleted ! StripeError {
return try await request(
c,
"DELETE",
`/v1/customers/${segment(id)}`,
null,
options,
j => try j as Deleted,
)
}
/// One page of customers, newest first.
export async fn list(
c: Client,
params: CustomerListParams = {},
options: RequestOptions = {},
): Page<Customer> ! StripeError {
return try await request(c, "GET", "/v1/customers", params, options, j => try j as Page<Customer>)
}
/// Every customer matching `params`, up to `max`.
export async fn listAll(
c: Client,
params: CustomerListParams = {},
options: RequestOptions = {},
max: int = 10000,
): Customer[] ! StripeError {
return try await collect(
c,
"/v1/customers",
params,
options,
j => try j as Page<Customer>,
x => x.id,
max,
)
}
/// Customers matching a search query (`email:"ana@example.com"`).
export async fn search(c: Client, query: string, options: RequestOptions = {}): SearchPage<
Customer,
> ! StripeError {
return try await request(
c,
"GET",
"/v1/customers/search",
{ query },
options,
j => try j as SearchPage<Customer>,
)
}

110
stripe/lib/form.weft Normal file
View file

@ -0,0 +1,110 @@
// Stripe's request encoding: `application/x-www-form-urlencoded` with nesting in brackets (`a[b][0]=c`). A `json` value's
// keys can't be listed in Weft, so this walks its JSON text, which keeps the fields in the order they were written.
import { parse, stringify } from "std/json"
fn isUnreserved(c: int): boolean {
return (c >= 48 && c <= 57) ||
(c >= 65 && c <= 90) ||
(c >= 97 && c <= 122) ||
c == 45 ||
c == 95 ||
c == 46 ||
c == 126
}
/// `text` percent-encoded as UTF-8, keeping only RFC 3986's unreserved characters.
export fn encodeComponent(text: string): string {
let mut out: string[] = []
for b of Bytes.fromUtf8(text).toList() {
if isUnreserved(b) {
out.push(String.fromCharCode(b))
} else {
out.push("%" + str(b, { radix: 16 }).toUpperCase().padStart(2, "0"))
}
}
return out.join("")
}
fn stringEnd(text: string, from: int): int {
let mut i = from + 1
while i < text.length {
let c = text.codeUnitAt(i) ?? 34
if c == 92 {
i += 2
} else if c == 34 {
return i + 1
} else {
i += 1
}
}
return i
}
fn pair(key: string, value: string): string {
let name = encodeComponent(key).replaceAll("%5B", "[").replaceAll("%5D", "]")
return `${name}=${encodeComponent(value)}`
}
fn child(key: string, name: string): string {
return key == "" ? name : `${key}[${name}]`
}
/// Writes the JSON value at `at` under `key`, and gives the position after it.
fn walk(text: string, at: int, key: string, mut out: string[]): int {
let c = text.codeUnitAt(at) ?? 0
if c == 123 {
let mut i = at + 1
while text.codeUnitAt(i) != 125 {
let end = stringEnd(text, i)
let name = try parse(text.slice(i, end)) else ""
let label = name is string ? name : ""
// `extra` holds fields this package doesn't model: they belong to the object it is written in.
i = walk(text, end + 1, label == "extra" ? key : child(key, label), &out)
if text.codeUnitAt(i) == 44 {
i += 1
}
}
return i + 1
}
if c == 91 {
let mut i = at + 1
let mut index = 0
while text.codeUnitAt(i) != 93 {
i = walk(text, i, `${key}[${index}]`, &out)
index += 1
if text.codeUnitAt(i) == 44 {
i += 1
}
}
return i + 1
}
if c == 34 {
let end = stringEnd(text, at)
let value = try parse(text.slice(at, end)) else ""
out.push(pair(key, value is string ? value : ""))
return end
}
let mut end = at
while end < text.length {
let u = text.codeUnitAt(end) ?? 44
if u == 44 || u == 125 || u == 93 {
break
}
end += 1
}
let literal = text.slice(at, end)
// A field left out is `null`: Stripe is sent nothing for it. To clear a field, send an empty string.
if literal != "null" {
out.push(pair(key, literal))
}
return end
}
/// `params` in Stripe's form encoding. `null` fields are left out.
export fn encodeForm(params: json): string {
let mut out: string[] = []
if params != null {
walk(stringify(params), 0, "", &out)
}
return out.join("&")
}

195
stripe/lib/sha256.weft Normal file
View file

@ -0,0 +1,195 @@
// SHA-256 and HMAC-SHA-256 (FIPS 180-4, RFC 2104), for verifying webhook signatures. Written here until `std/crypto` is
// on main: when it is, this file goes and `webhooks.weft` calls `hmac` and `timingSafeEqual` from there.
let K: i64[] = [
0x428a2f98,
0x71374491,
0xb5c0fbcf,
0xe9b5dba5,
0x3956c25b,
0x59f111f1,
0x923f82a4,
0xab1c5ed5,
0xd807aa98,
0x12835b01,
0x243185be,
0x550c7dc3,
0x72be5d74,
0x80deb1fe,
0x9bdc06a7,
0xc19bf174,
0xe49b69c1,
0xefbe4786,
0x0fc19dc6,
0x240ca1cc,
0x2de92c6f,
0x4a7484aa,
0x5cb0a9dc,
0x76f988da,
0x983e5152,
0xa831c66d,
0xb00327c8,
0xbf597fc7,
0xc6e00bf3,
0xd5a79147,
0x06ca6351,
0x14292967,
0x27b70a85,
0x2e1b2138,
0x4d2c6dfc,
0x53380d13,
0x650a7354,
0x766a0abb,
0x81c2c92e,
0x92722c85,
0xa2bfe8a1,
0xa81a664b,
0xc24b8b70,
0xc76c51a3,
0xd192e819,
0xd6990624,
0xf40e3585,
0x106aa070,
0x19a4c116,
0x1e376c08,
0x2748774c,
0x34b0bcb5,
0x391c0cb3,
0x4ed8aa4a,
0x5b9cca4f,
0x682e6ff3,
0x748f82ee,
0x78a5636f,
0x84c87814,
0x8cc70208,
0x90befffa,
0xa4506ceb,
0xbef9a3f7,
0xc67178f2,
]
/// The low 32 bits. Words are held in an `i64`, whose sums can't overflow, and masked: SHA-256's additions wrap, and an
/// `int`'s panic.
let WORD: i64 = 0xffffffff
fn shr(x: i64, n: int): i64 {
return x >>> n
}
fn rotr(x: i64, n: int): i64 {
return (shr(x, n) | (x << (32 - n))) & WORD
}
fn compress(mut h: i64[], m: int[], at: int) {
let mut w: i64[] = []
for t of 0..16 {
let i = at + t * 4
w.push(i64((m[i] << 24) | (m[i + 1] << 16) | (m[i + 2] << 8) | m[i + 3]) & WORD)
}
for t of 16..64 {
let a = w[t - 15]
let b = w[t - 2]
let s0 = rotr(a, 7) ^ rotr(a, 18) ^ shr(a, 3)
let s1 = rotr(b, 17) ^ rotr(b, 19) ^ shr(b, 10)
w.push((w[t - 16] + s0 + w[t - 7] + s1) & WORD)
}
let mut a = h[0]
let mut b = h[1]
let mut c = h[2]
let mut d = h[3]
let mut e = h[4]
let mut f = h[5]
let mut g = h[6]
let mut k = h[7]
for t of 0..64 {
let s1 = rotr(e, 6) ^ rotr(e, 11) ^ rotr(e, 25)
let ch = (e & f) ^ (~e & g & WORD)
let t1 = (k + s1 + ch + K[t] + w[t]) & WORD
let s0 = rotr(a, 2) ^ rotr(a, 13) ^ rotr(a, 22)
let maj = (a & b) ^ (a & c) ^ (b & c)
let t2 = (s0 + maj) & WORD
k = g
g = f
f = e
e = (d + t1) & WORD
d = c
c = b
b = a
a = (t1 + t2) & WORD
}
let sums = [a, b, c, d, e, f, g, k]
for i of 0..8 {
h[i] = (h[i] + sums[i]) & WORD
}
}
/// The byte of `word` that `shift` bits down.
fn byteOf(word: i64, shift: int): int {
return try int(shr(word, shift) & 255) else 0
}
/// The SHA-256 digest of `data`.
export fn sha256(data: Bytes): Bytes {
let mut m = data.toList()
let length = i64(m.length)
m.push(0x80)
while m.length % 64 != 56 {
m.push(0)
}
// The length in bits, as 64 bits big-endian.
let bits = length * 8
let high = shr(bits, 32) & WORD
let low = bits & WORD
for shift of [24, 16, 8, 0] {
m.push(byteOf(high, shift))
}
for shift of [24, 16, 8, 0] {
m.push(byteOf(low, shift))
}
let mut h: i64[] = [
0x6a09e667,
0xbb67ae85,
0x3c6ef372,
0xa54ff53a,
0x510e527f,
0x9b05688c,
0x1f83d9ab,
0x5be0cd19,
]
let mut at = 0
while at < m.length {
compress(&h, m, at)
at += 64
}
let mut out: int[] = []
for word of h {
for shift of [24, 16, 8, 0] {
out.push(byteOf(word, shift))
}
}
return Bytes.of(out)
}
/// HMAC-SHA-256 of `data` with `key`.
export fn hmacSha256(key: Bytes, data: Bytes): Bytes {
let mut block = (key.length > 64 ? sha256(key) : key).toList()
while block.length < 64 {
block.push(0)
}
let inner = Bytes.of(block.map(b => b ^ 0x36)).concat(data)
let outer = Bytes.of(block.map(b => b ^ 0x5c))
return sha256(outer.concat(sha256(inner)))
}
/// Whether two byte strings are equal, looking at every byte whatever it finds.
export fn constantTimeEqual(a: Bytes, b: Bytes): boolean {
if a.length != b.length {
return false
}
let x = a.toList()
let y = b.toList()
let mut diff = 0
for i of 0..x.length {
diff = diff | (x[i] ^ y[i])
}
return diff == 0
}

235
stripe/payment_intents.weft Normal file
View file

@ -0,0 +1,235 @@
// Payment intents: one attempt to collect a payment.
import { collect, request, segment } from "./stripe"
import type { Client, Page, RequestOptions, SearchPage, StripeError } from "./stripe"
/// A payment intent. `customer`, `payment_method` and `latest_charge` are expandable: `idOf` gives their ids.
export type PaymentIntent = {
id: string,
amount: float,
currency: string,
status: string,
amount_capturable?: float,
amount_received?: float,
application_fee_amount?: float,
capture_method?: string,
client_secret?: string,
confirmation_method?: string,
customer?: json,
description?: string,
last_payment_error?: json,
latest_charge?: json,
metadata?: json,
next_action?: json,
on_behalf_of?: json,
payment_method?: json,
payment_method_types?: string[],
receipt_email?: string,
statement_descriptor?: string,
transfer_data?: json,
transfer_group?: string,
canceled_at?: float,
cancellation_reason?: string,
created?: float,
livemode?: boolean,
}
/// A new payment intent. `amount` is in the currency's smallest unit. `extra` holds any field this type doesn't list.
export type PaymentIntentParams = {
amount: float,
currency: string,
customer?: string,
description?: string,
metadata?: Map<string, string>,
payment_method?: string,
payment_method_types?: string[],
automatic_payment_methods?: { enabled: boolean, allow_redirects?: string },
capture_method?: string,
confirm?: boolean,
return_url?: string,
receipt_email?: string,
setup_future_usage?: string,
statement_descriptor?: string,
application_fee_amount?: float,
on_behalf_of?: string,
transfer_data?: { destination: string, amount?: float },
transfer_group?: string,
extra?: json,
}
/// The fields of a payment intent that can change before it is confirmed.
export type PaymentIntentUpdateParams = {
amount?: float,
currency?: string,
customer?: string,
description?: string,
metadata?: Map<string, string>,
payment_method?: string,
receipt_email?: string,
application_fee_amount?: float,
extra?: json,
}
/// What to list.
export type PaymentIntentListParams = {
customer?: string,
limit?: float,
starting_after?: string,
ending_before?: string,
extra?: json,
}
/// Creates a payment intent.
///
/// ```weft
/// let intent = try await paymentIntents.create(stripe, {
/// amount: 1999,
/// currency: "eur",
/// automatic_payment_methods: { enabled: true },
/// }, { idempotencyKey: `order-${order.id}` })
/// ```
export async fn create(
c: Client,
params: PaymentIntentParams,
options: RequestOptions = {},
): PaymentIntent ! StripeError {
return try await request(
c,
"POST",
"/v1/payment_intents",
params,
options,
j => try j as PaymentIntent,
)
}
/// Gets a payment intent by id.
export async fn retrieve(
c: Client,
id: string,
options: RequestOptions = {},
): PaymentIntent ! StripeError {
return try await request(
c,
"GET",
`/v1/payment_intents/${segment(id)}`,
null,
options,
j => try j as PaymentIntent,
)
}
/// Changes the fields given.
export async fn update(
c: Client,
id: string,
params: PaymentIntentUpdateParams,
options: RequestOptions = {},
): PaymentIntent ! StripeError {
return try await request(
c,
"POST",
`/v1/payment_intents/${segment(id)}`,
params,
options,
j => try j as PaymentIntent,
)
}
/// Confirms that the customer intends to pay. `params` may name a `payment_method` or a `return_url`.
export async fn confirm(
c: Client,
id: string,
params: json = null,
options: RequestOptions = {},
): PaymentIntent ! StripeError {
return try await request(
c,
"POST",
`/v1/payment_intents/${segment(id)}/confirm`,
params,
options,
j => try j as PaymentIntent,
)
}
/// Captures an authorised payment, all of it unless `params` gives an `amount_to_capture`.
export async fn capture(
c: Client,
id: string,
params: json = null,
options: RequestOptions = {},
): PaymentIntent ! StripeError {
return try await request(
c,
"POST",
`/v1/payment_intents/${segment(id)}/capture`,
params,
options,
j => try j as PaymentIntent,
)
}
/// Cancels a payment intent that hasn't succeeded.
export async fn cancel(
c: Client,
id: string,
params: json = null,
options: RequestOptions = {},
): PaymentIntent ! StripeError {
return try await request(
c,
"POST",
`/v1/payment_intents/${segment(id)}/cancel`,
params,
options,
j => try j as PaymentIntent,
)
}
/// One page of payment intents, newest first.
export async fn list(
c: Client,
params: PaymentIntentListParams = {},
options: RequestOptions = {},
): Page<PaymentIntent> ! StripeError {
return try await request(
c,
"GET",
"/v1/payment_intents",
params,
options,
j => try j as Page<PaymentIntent>,
)
}
/// Every payment intent matching `params`, up to `max`.
export async fn listAll(
c: Client,
params: PaymentIntentListParams = {},
options: RequestOptions = {},
max: int = 10000,
): PaymentIntent[] ! StripeError {
return try await collect(
c,
"/v1/payment_intents",
params,
options,
j => try j as Page<PaymentIntent>,
x => x.id,
max,
)
}
/// Payment intents matching a search query (`status:"succeeded"`).
export async fn search(c: Client, query: string, options: RequestOptions = {}): SearchPage<
PaymentIntent,
> ! StripeError {
return try await request(
c,
"GET",
"/v1/payment_intents/search",
{ query },
options,
j => try j as SearchPage<PaymentIntent>,
)
}

139
stripe/prices.weft Normal file
View file

@ -0,0 +1,139 @@
// Prices: what a product costs.
import { collect, request, segment } from "./stripe"
import type { Client, Page, RequestOptions, StripeError } from "./stripe"
/// How often a recurring price is charged.
export type Recurring = {
interval: string,
interval_count?: float,
usage_type?: string,
meter?: string,
}
/// A price. `unit_amount` is in the currency's smallest unit (cents). `product` is expandable: `idOf` gives its id.
export type Price = {
id: string,
active?: boolean,
currency?: string,
unit_amount?: float,
unit_amount_decimal?: string,
billing_scheme?: string,
lookup_key?: string,
nickname?: string,
product?: json,
recurring?: Recurring,
tax_behavior?: string,
type?: string,
metadata?: json,
created?: float,
livemode?: boolean,
}
/// A new price. `extra` holds any field this type doesn't list.
export type PriceParams = {
currency: string,
unit_amount?: float,
unit_amount_decimal?: string,
product?: string,
product_data?: { name: string, active?: boolean, metadata?: Map<string, string> },
recurring?: { interval: string, interval_count?: float },
lookup_key?: string,
transfer_lookup_key?: boolean,
nickname?: string,
active?: boolean,
tax_behavior?: string,
metadata?: Map<string, string>,
extra?: json,
}
/// The fields of a price that can change after it is made.
export type PriceUpdateParams = {
active?: boolean,
lookup_key?: string,
transfer_lookup_key?: boolean,
nickname?: string,
tax_behavior?: string,
metadata?: Map<string, string>,
extra?: json,
}
/// What to list. `lookup_keys` finds prices by the stable names you gave them.
export type PriceListParams = {
active?: boolean,
currency?: string,
product?: string,
type?: string,
lookup_keys?: string[],
recurring?: { interval?: string, usage_type?: string },
limit?: float,
starting_after?: string,
ending_before?: string,
extra?: json,
}
/// Creates a price.
export async fn create(
c: Client,
params: PriceParams,
options: RequestOptions = {},
): Price ! StripeError {
return try await request(c, "POST", "/v1/prices", params, options, j => try j as Price)
}
/// Gets a price by id.
export async fn retrieve(c: Client, id: string, options: RequestOptions = {}): Price ! StripeError {
return try await request(
c,
"GET",
`/v1/prices/${segment(id)}`,
null,
options,
j => try j as Price,
)
}
/// Changes the fields given.
export async fn update(
c: Client,
id: string,
params: PriceUpdateParams,
options: RequestOptions = {},
): Price ! StripeError {
return try await request(
c,
"POST",
`/v1/prices/${segment(id)}`,
params,
options,
j => try j as Price,
)
}
/// One page of prices.
///
/// ```weft
/// let found = try await prices.list(stripe, { lookup_keys: ["base-monthly"], active: true })
/// ```
export async fn list(c: Client, params: PriceListParams = {}, options: RequestOptions = {}): Page<
Price,
> ! StripeError {
return try await request(c, "GET", "/v1/prices", params, options, j => try j as Page<Price>)
}
/// Every price matching `params`, up to `max`.
export async fn listAll(
c: Client,
params: PriceListParams = {},
options: RequestOptions = {},
max: int = 10000,
): Price[] ! StripeError {
return try await collect(
c,
"/v1/prices",
params,
options,
j => try j as Page<Price>,
x => x.id,
max,
)
}

125
stripe/products.weft Normal file
View file

@ -0,0 +1,125 @@
// Products: what you sell.
import { collect, request, segment } from "./stripe"
import type { Client, Deleted, Page, RequestOptions, StripeError } from "./stripe"
/// A product. `default_price` is expandable: `idOf` gives its id.
export type Product = {
id: string,
name?: string,
active?: boolean,
description?: string,
images?: string[],
metadata?: json,
default_price?: json,
tax_code?: json,
unit_label?: string,
url?: string,
created?: float,
updated?: float,
livemode?: boolean,
}
/// The fields of a product you can set. `extra` holds any field this type doesn't list.
export type ProductParams = {
name?: string,
id?: string,
active?: boolean,
description?: string,
images?: string[],
metadata?: Map<string, string>,
default_price?: string,
tax_code?: string,
unit_label?: string,
url?: string,
extra?: json,
}
/// What to list.
export type ProductListParams = {
active?: boolean,
ids?: string[],
url?: string,
limit?: float,
starting_after?: string,
ending_before?: string,
extra?: json,
}
/// Creates a product.
export async fn create(
c: Client,
params: ProductParams,
options: RequestOptions = {},
): Product ! StripeError {
return try await request(c, "POST", "/v1/products", params, options, j => try j as Product)
}
/// Gets a product by id.
export async fn retrieve(
c: Client,
id: string,
options: RequestOptions = {},
): Product ! StripeError {
return try await request(
c,
"GET",
`/v1/products/${segment(id)}`,
null,
options,
j => try j as Product,
)
}
/// Changes the fields given.
export async fn update(
c: Client,
id: string,
params: ProductParams,
options: RequestOptions = {},
): Product ! StripeError {
return try await request(
c,
"POST",
`/v1/products/${segment(id)}`,
params,
options,
j => try j as Product,
)
}
/// Deletes a product that has no prices.
export async fn remove(c: Client, id: string, options: RequestOptions = {}): Deleted ! StripeError {
return try await request(
c,
"DELETE",
`/v1/products/${segment(id)}`,
null,
options,
j => try j as Deleted,
)
}
/// One page of products.
export async fn list(c: Client, params: ProductListParams = {}, options: RequestOptions = {}): Page<
Product,
> ! StripeError {
return try await request(c, "GET", "/v1/products", params, options, j => try j as Page<Product>)
}
/// Every product matching `params`, up to `max`.
export async fn listAll(
c: Client,
params: ProductListParams = {},
options: RequestOptions = {},
max: int = 10000,
): Product[] ! StripeError {
return try await collect(
c,
"/v1/products",
params,
options,
j => try j as Page<Product>,
x => x.id,
max,
)
}

90
stripe/refunds.weft Normal file
View file

@ -0,0 +1,90 @@
// Refunds.
import { collect, request, segment } from "./stripe"
import type { Client, Page, RequestOptions, StripeError } from "./stripe"
/// A refund. `charge` and `payment_intent` are expandable: `idOf` gives their ids.
export type Refund = {
id: string,
amount: float,
currency: string,
status?: string,
reason?: string,
charge?: json,
payment_intent?: json,
failure_reason?: string,
receipt_number?: string,
metadata?: json,
created?: float,
}
/// A new refund of a `payment_intent` or a `charge`: all of it unless `amount` is given.
export type RefundParams = {
payment_intent?: string,
charge?: string,
amount?: float,
reason?: "duplicate" | "fraudulent" | "requested_by_customer",
metadata?: Map<string, string>,
refund_application_fee?: boolean,
reverse_transfer?: boolean,
extra?: json,
}
/// What to list.
export type RefundListParams = {
payment_intent?: string,
charge?: string,
limit?: float,
starting_after?: string,
ending_before?: string,
extra?: json,
}
/// Refunds a payment.
export async fn create(
c: Client,
params: RefundParams,
options: RequestOptions = {},
): Refund ! StripeError {
return try await request(c, "POST", "/v1/refunds", params, options, j => try j as Refund)
}
/// Gets a refund by id.
export async fn retrieve(
c: Client,
id: string,
options: RequestOptions = {},
): Refund ! StripeError {
return try await request(
c,
"GET",
`/v1/refunds/${segment(id)}`,
null,
options,
j => try j as Refund,
)
}
/// One page of refunds, newest first.
export async fn list(c: Client, params: RefundListParams = {}, options: RequestOptions = {}): Page<
Refund,
> ! StripeError {
return try await request(c, "GET", "/v1/refunds", params, options, j => try j as Page<Refund>)
}
/// Every refund matching `params`, up to `max`.
export async fn listAll(
c: Client,
params: RefundListParams = {},
options: RequestOptions = {},
max: int = 10000,
): Refund[] ! StripeError {
return try await collect(
c,
"/v1/refunds",
params,
options,
j => try j as Page<Refund>,
x => x.id,
max,
)
}

View file

@ -0,0 +1,31 @@
// Runs packages/integration/stripe against stripe-mock (https://github.com/stripe/stripe-mock), which must be on the PATH or
// named by STRIPE_MOCK. It needs no network and no Stripe account.
import { spawn, spawnSync } from "node:child_process"
import { dirname, join } from "node:path"
import { fileURLToPath } from "node:url"
const root = join(dirname(fileURLToPath(import.meta.url)), "..")
const weft = process.env.WEFT_BIN ?? join(root, "../../core/target/release/weft")
const port = 12111 + (process.pid % 1000)
const mock = spawn(process.env.STRIPE_MOCK ?? "stripe-mock", ["-http-port", String(port), "-https-port", "0"], { stdio: "ignore" })
let status = 1
try {
for (let i = 0; ; i++) {
const up = await fetch(`http://127.0.0.1:${port}/v1/customers`, { headers: { authorization: "Bearer sk_test_1" } }).then(() => true, () => false)
if (up) break
if (i > 100) throw new Error("stripe-mock didn't start")
await new Promise((r) => setTimeout(r, 100))
}
spawnSync(weft, ["install"], { cwd: join(root, "../integration/stripe"), stdio: "inherit" })
const r = spawnSync(weft, ["run", "."], {
cwd: join(root, "../integration/stripe"),
encoding: "utf8",
env: { ...process.env, STRIPE_MOCK_URL: `http://127.0.0.1:${port}`, WEFT_DEADLINE_MS: "0" },
})
process.stdout.write(r.stdout)
process.stderr.write(r.stderr)
status = r.status === 0 ? 0 : 1
} finally {
mock.kill()
}
process.exit(status)

316
stripe/stripe.weft Normal file
View file

@ -0,0 +1,316 @@
// `@weft/stripe`: the client, its errors and the calls every resource module is built on.
import { fetch } from "std/http"
import { parse } from "std/json"
import { floor, min } from "std/math"
import { random, uuid } from "std/random"
import { sleep } from "std/time"
import { encodeComponent, encodeForm } from "./lib/form"
/// The Stripe API version this package's types are written against.
export let API_VERSION = "2026-10-28.endive"
/// A configured client. Make one with `client`; it holds no connection and can be shared freely.
export type Client = {
apiKey: string,
apiVersion: string,
baseUrl: string,
maxRetries: float,
timeout: float,
stripeAccount: string?,
}
/// Options for `client`. `maxRetries` (2) is how often a request that failed in a way that is safe to repeat is sent again;
/// `timeout` (80,000) is in milliseconds; `stripeAccount` makes every call on behalf of a connected account.
export type ClientOptions = {
apiVersion?: string,
baseUrl?: string,
maxRetries?: float,
timeout?: float,
stripeAccount?: string,
}
/// Options for one call. `idempotencyKey` makes a retried `POST` safe to repeat (one is generated when left out);
/// `stripeAccount` makes this call on behalf of a connected account; `expand` lists the fields to expand.
export type RequestOptions = {
idempotencyKey?: string,
stripeAccount?: string,
expand?: string[],
timeout?: float,
maxRetries?: float,
}
/// A client for an API key. Keep the key on the server: read it from the environment.
///
/// ```weft
/// import { client } from "@weft/stripe"
/// import { env } from "std/process"
///
/// let stripe = client(env("STRIPE_SECRET_KEY")!)
/// ```
export fn client(apiKey: string, options: ClientOptions = {}): Client {
return {
apiKey,
apiVersion: options.apiVersion ?? API_VERSION,
baseUrl: options.baseUrl ?? "https://api.stripe.com",
maxRetries: options.maxRetries ?? 2,
timeout: options.timeout ?? 80000,
stripeAccount: options.stripeAccount,
}
}
/// What Stripe said about a request it refused.
export type Failure = {
message: string,
status: float,
type: string,
code: string?,
decline_code: string?,
param: string?,
doc_url: string?,
charge: string?,
payment_intent: string?,
request_id: string?,
}
/// Why a call failed. `CardError` is the customer's card being declined; `ConnectionFailed` means Stripe was never
/// reached, so the outcome of a `POST` is unknown; `BadResponse` means Stripe answered with something this package can't
/// read.
export type StripeError =
| CardError { failure: Failure }
| InvalidRequest { failure: Failure }
| AuthenticationFailed { failure: Failure }
| PermissionDenied { failure: Failure }
| RateLimited { failure: Failure }
| IdempotencyConflict { failure: Failure }
| ApiFailure { failure: Failure }
| ConnectionFailed { message: string }
| BadResponse { message: string, request_id: string? }
/// A sentence saying what went wrong, for logs.
export fn errorMessage(error: StripeError): string {
return match error {
CardError { failure } | InvalidRequest { failure } | AuthenticationFailed {
failure,
} | PermissionDenied { failure } | RateLimited { failure } | IdempotencyConflict {
failure,
} | ApiFailure { failure } => failure.message,
ConnectionFailed { message } | BadResponse { message } => message,
}
}
/// One page of a list.
export type Page<T> = { data: T[], has_more: boolean, url?: string }
/// One page of a search. `next_page` is what to pass as `page` for the next one.
export type SearchPage<T> = {
data: T[],
has_more: boolean,
next_page?: string,
total_count?: float,
}
/// What deleting an object answers.
export type Deleted = { id: string, deleted: boolean }
/// A postal address.
export type Address = {
city?: string,
country?: string,
line1?: string,
line2?: string,
postal_code?: string,
state?: string,
}
/// The id of an expandable field: the field itself when it wasn't expanded, its `id` when it was.
///
/// ```weft
/// let customerId = idOf(session.customer)
/// ```
export fn idOf(field: json): string? {
if field is string {
return field
}
let id = field.id
return id is string ? id : null
}
/// A value from an object's `metadata`.
export fn metadata(field: json, key: string): string? {
let value = field[key]
return value is string ? value : null
}
fn text(value: json): string? {
return value is string ? value : null
}
/// The error for an answer that isn't a success.
export fn failureFrom(status: int, body: json, requestId: string?): StripeError {
let e = body.error
let kind = text(e.type) ?? ""
let failure: Failure = {
message: text(e.message) ?? `Stripe answered ${status}`,
status,
type: kind,
code: text(e.code),
decline_code: text(e.decline_code),
param: text(e.param),
doc_url: text(e.doc_url),
charge: text(e.charge),
payment_intent: idOf(e.payment_intent),
request_id: requestId,
}
if kind == "card_error" || status == 402 {
return CardError { failure }
}
if kind == "idempotency_error" {
return IdempotencyConflict { failure }
}
return match status {
400 | 404 => InvalidRequest { failure },
401 => AuthenticationFailed { failure },
403 => PermissionDenied { failure },
429 => RateLimited { failure },
_ => ApiFailure { failure },
}
}
/// Stripe's own advice on repeating a request wins; otherwise conflicts and server errors are repeated.
fn shouldRetry(status: float, advice: string?): boolean {
if advice == "false" {
return false
}
return advice == "true" || status == 409 || status >= 500
}
/// How long to wait before attempt `attempt + 1`: exponential with jitter, or what `Retry-After` asks for up to a minute.
fn retryDelay(attempt: int, retryAfter: string?): float {
let asked = retryAfter != null ? (try parse(retryAfter) else null) : null
if asked is float && asked >= 0 && asked <= 60 {
return asked * 1000
}
let base = min(500 * 2 ** attempt, 8000)
return floor(base * (0.75 + random() * 0.25))
}
/// An id made safe for a path.
export fn segment(id: string): string {
return encodeComponent(id)
}
/// Sends one request and decodes the answer: what every function in this package calls. Use it for the parts of Stripe's
/// API this package doesn't model.
///
/// ```weft
/// let balance = try await request(stripe, "GET", "/v1/balance", null, {}, j => try j as Balance)
/// ```
export async fn request<T>(
c: Client,
method: "GET" | "POST" | "DELETE",
path: string,
params: json,
options: RequestOptions,
escaping decode: (value: json) => T ! CastError,
): T ! StripeError {
let mut fields = params
let expand = options.expand
if expand != null && expand.length > 0 {
if fields == null {
fields = try parse("{}") else null
}
fields["expand"] = expand
}
let form = encodeForm(fields)
let write = method == "POST"
let url = !write && form != "" ? `${c.baseUrl}${path}?${form}` : `${c.baseUrl}${path}`
let retries = options.maxRetries ?? c.maxRetries
let mut headers: [string, string][] = [
["authorization", `Bearer ${c.apiKey}`],
["stripe-version", c.apiVersion],
["user-agent", "weft-stripe/0.1.0"],
["accept", "application/json"],
]
if write {
headers.push(["content-type", "application/x-www-form-urlencoded"])
// One key for every attempt, so a request Stripe did receive isn't carried out twice.
let key = options.idempotencyKey ?? (retries > 0 ? uuid() : null)
if key != null {
headers.push(["idempotency-key", key])
}
}
let account = options.stripeAccount ?? c.stripeAccount
if account != null {
headers.push(["stripe-account", account])
}
let timeout = options.timeout ?? c.timeout
let mut attempt = 0
while true {
let res, sent = try await fetch(url, { method, headers, body: write ? form : null, timeout })
if sent != null {
if attempt < retries {
await sleep(retryDelay(attempt, null))
attempt += 1
continue
}
fail ConnectionFailed { message: `could not reach Stripe: ${sent.message}` }
}
let requestId = res.headers.get("request-id")
let body, read = try await res.text()
if read != null {
if attempt < retries {
await sleep(retryDelay(attempt, null))
attempt += 1
continue
}
fail ConnectionFailed { message: `the answer from Stripe broke off: ${read.message}` }
}
let parsed = try parse(body) else null
if res.status >= 200 && res.status < 300 {
let value, wrong = try decode(parsed)
if wrong != null {
fail BadResponse {
message: `unexpected answer from ${path}: ${wrong.path} isn't ${wrong.expected}`,
request_id: requestId,
}
}
return value
}
if attempt < retries && shouldRetry(res.status, res.headers.get("stripe-should-retry")) {
await sleep(retryDelay(attempt, res.headers.get("retry-after")))
attempt += 1
continue
}
fail failureFrom(res.status.truncToInt(), parsed, requestId)
}
}
/// Every item of a list, following its pages up to `max` items.
export async fn collect<T>(
c: Client,
path: string,
params: json,
options: RequestOptions,
escaping decode: (value: json) => Page<T> ! CastError,
escaping id: (item: T) => string,
max: int,
): T[] ! StripeError {
let mut items: T[] = []
let mut after: string? = null
while items.length < max {
let mut fields = params
if after != null {
if fields == null {
fields = try parse("{}") else null
}
fields["starting_after"] = after
}
let page = try await request(c, "GET", path, fields, options, decode)
items.push(...page.data)
if !page.has_more || page.data.length == 0 {
break
}
after = id(page.data[page.data.length - 1])
}
return items.slice(0, max)
}

215
stripe/subscriptions.weft Normal file
View file

@ -0,0 +1,215 @@
// Subscriptions: recurring payments.
import { collect, request, segment } from "./stripe"
import type { Client, Page, RequestOptions, SearchPage, StripeError } from "./stripe"
import type { Price } from "./prices"
/// One price on a subscription. On current API versions the billing period is on the item.
export type SubscriptionItem = {
id: string,
price?: Price,
quantity?: float,
metadata?: json,
current_period_start?: float,
current_period_end?: float,
created?: float,
}
/// A subscription. `customer`, `latest_invoice` and `default_payment_method` are expandable: `idOf` gives their ids.
export type Subscription = {
id: string,
status: string,
customer: json,
items: { data: SubscriptionItem[], has_more?: boolean },
metadata?: json,
currency?: string,
cancel_at?: float,
cancel_at_period_end?: boolean,
canceled_at?: float,
collection_method?: string,
default_payment_method?: json,
discounts?: json,
ended_at?: float,
latest_invoice?: json,
pending_setup_intent?: json,
start_date?: float,
trial_end?: float,
trial_start?: float,
created?: float,
livemode?: boolean,
}
/// A price on a new subscription.
export type SubscriptionItemParams = {
price?: string,
quantity?: float,
metadata?: Map<string, string>,
extra?: json,
}
/// A new subscription. `extra` holds any field this type doesn't list.
export type SubscriptionParams = {
customer: string,
items: SubscriptionItemParams[],
metadata?: Map<string, string>,
cancel_at_period_end?: boolean,
collection_method?: string,
days_until_due?: float,
default_payment_method?: string,
discounts?: { coupon?: string, promotion_code?: string }[],
payment_behavior?: string,
proration_behavior?: string,
trial_end?: float,
trial_period_days?: float,
extra?: json,
}
/// A change to one item. Give `id` to change an item, leave it out to add one, and set `deleted` to remove it.
export type SubscriptionItemUpdate = {
id?: string,
price?: string,
quantity?: float,
deleted?: boolean,
metadata?: Map<string, string>,
extra?: json,
}
/// The fields of a subscription that can change.
export type SubscriptionUpdateParams = {
items?: SubscriptionItemUpdate[],
metadata?: Map<string, string>,
cancel_at_period_end?: boolean,
default_payment_method?: string,
discounts?: { coupon?: string, promotion_code?: string, discount?: string }[],
payment_behavior?: string,
proration_behavior?: string,
trial_end?: float,
extra?: json,
}
/// What to list. `status` also takes `all`, which includes canceled subscriptions.
export type SubscriptionListParams = {
customer?: string,
price?: string,
status?: string,
limit?: float,
starting_after?: string,
ending_before?: string,
extra?: json,
}
/// Creates a subscription.
export async fn create(
c: Client,
params: SubscriptionParams,
options: RequestOptions = {},
): Subscription ! StripeError {
return try await request(
c,
"POST",
"/v1/subscriptions",
params,
options,
j => try j as Subscription,
)
}
/// Gets a subscription by id.
export async fn retrieve(
c: Client,
id: string,
options: RequestOptions = {},
): Subscription ! StripeError {
return try await request(
c,
"GET",
`/v1/subscriptions/${segment(id)}`,
null,
options,
j => try j as Subscription,
)
}
/// Changes the fields given.
export async fn update(
c: Client,
id: string,
params: SubscriptionUpdateParams,
options: RequestOptions = {},
): Subscription ! StripeError {
return try await request(
c,
"POST",
`/v1/subscriptions/${segment(id)}`,
params,
options,
j => try j as Subscription,
)
}
/// Cancels a subscription now. To cancel when the period ends, `update` with `cancel_at_period_end: true`.
export async fn cancel(
c: Client,
id: string,
options: RequestOptions = {},
): Subscription ! StripeError {
return try await request(
c,
"DELETE",
`/v1/subscriptions/${segment(id)}`,
null,
options,
j => try j as Subscription,
)
}
/// One page of subscriptions, newest first. Canceled ones are left out unless `status` asks for them.
export async fn list(
c: Client,
params: SubscriptionListParams = {},
options: RequestOptions = {},
): Page<Subscription> ! StripeError {
return try await request(
c,
"GET",
"/v1/subscriptions",
params,
options,
j => try j as Page<Subscription>,
)
}
/// Every subscription matching `params`, up to `max`.
export async fn listAll(
c: Client,
params: SubscriptionListParams = {},
options: RequestOptions = {},
max: int = 10000,
): Subscription[] ! StripeError {
return try await collect(
c,
"/v1/subscriptions",
params,
options,
j => try j as Page<Subscription>,
x => x.id,
max,
)
}
/// Subscriptions matching a search query (`status:"active" AND metadata["site"]:"a1"`). `page` is the `next_page` of the
/// page before.
export async fn search(
c: Client,
query: string,
page: string? = null,
options: RequestOptions = {},
): SearchPage<Subscription> ! StripeError {
return try await request(
c,
"GET",
"/v1/subscriptions/search",
{ query, page },
options,
j => try j as SearchPage<Subscription>,
)
}

View file

@ -0,0 +1,220 @@
// `weft test tests/stripe.test.weft`: what needs no server. packages/integration/stripe runs the rest against stripe-mock.
import { assert, assertEq } from "std/testing"
import { encodeComponent, encodeForm } from "../lib/form"
import { constantTimeEqual, hmacSha256, sha256 } from "../lib/sha256"
import { client, errorMessage, failureFrom, idOf, metadata } from "../stripe"
import { signature, verify } from "../webhooks"
import type { WebhookError } from "../webhooks"
import { parse } from "std/json"
fn hex(s: string): Bytes {
return try Bytes.fromHex(s) else Bytes.of([])
}
// ---- SHA-256 and HMAC: FIPS 180-4's and RFC 4231's own vectors ----
test "sha256 vectors" {
assertEq(
sha256(Bytes.fromUtf8("")).toHex(),
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
)
assertEq(
sha256(Bytes.fromUtf8("abc")).toHex(),
"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad",
)
assertEq(
sha256(Bytes.fromUtf8("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq")).toHex(),
"248d6a61d20638b8e5c026930c3e6039a33ce45964ff2167f6ecedd419db06c1",
)
assertEq(
sha256(Bytes.fromUtf8("a".repeat(1000))).toHex(),
"41edece42d63e8d9bf515a9ba6932e1c20cbc9f5a5d134645adb5db1b9737ea3",
)
}
test "padding at every block boundary" {
// Lengths around 55, 56 and 64 are where the padding spills into another block.
assertEq(
sha256(Bytes.fromUtf8("a".repeat(55))).toHex(),
"9f4390f8d30c2dd92ec9f095b65e2b9ae9b0a925a5258e241c9f1e910f734318",
)
assertEq(
sha256(Bytes.fromUtf8("a".repeat(56))).toHex(),
"b35439a4ac6f0948b6d6f9e3c6af0f5f590ce20f1bde7090ef7970686ec6738a",
)
assertEq(
sha256(Bytes.fromUtf8("a".repeat(64))).toHex(),
"ffe054fe7ae0cb6dc65c3af9b61d5209f439851db43d0ba5997337df154668eb",
)
}
test "hmac vectors (RFC 4231)" {
assertEq(
hmacSha256(hex("0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b"), Bytes.fromUtf8("Hi There")).toHex(),
"b0344c61d8db38535ca8afceaf0bf12b881dc200c9833da726e9376c2e32cff7",
)
assertEq(
hmacSha256(Bytes.fromUtf8("Jefe"), Bytes.fromUtf8("what do ya want for nothing?")).toHex(),
"5bdcc146bf60754e6a042426089575c75a003f089d2739839dec58b964ec3843",
)
let longKey = Bytes.of(Array131())
assertEq(
hmacSha256(
longKey,
Bytes.fromUtf8("Test Using Larger Than Block-Size Key - Hash Key First"),
).toHex(),
"60e431591ee0b67f0d8a26aacbf5b77f8e0bc6213728c5140546040f0ee37f54",
)
}
fn Array131(): int[] {
let mut out: int[] = []
for i of 0..131 {
out.push(0xaa)
}
return out
}
test "constant-time comparison" {
assert(constantTimeEqual(Bytes.fromUtf8("abc"), Bytes.fromUtf8("abc")))
assert(!constantTimeEqual(Bytes.fromUtf8("abc"), Bytes.fromUtf8("abd")))
assert(!constantTimeEqual(Bytes.fromUtf8("abc"), Bytes.fromUtf8("abcd")))
}
// ---- the form encoding ----
test "form encoding nests in brackets and leaves null out" {
let params: json = {
mode: "payment",
amount: 1999,
confirm: true,
skipped: null,
metadata: Map<string, string>([["sale id", "a&b=c"]]),
line_items: [
{ quantity: 2, price_data: { currency: "eur", product_data: { name: "Vstopnica č" } } },
],
}
assertEq(
encodeForm(params),
"mode=payment&amount=1999&confirm=true&metadata[sale%20id]=a%26b%3Dc&line_items[0][quantity]=2&line_items[0][price_data][currency]=eur&line_items[0][price_data][product_data][name]=Vstopnica%20%C4%8D",
)
}
test "form encoding: strings with quotes and escapes, empty strings, extra" {
let params: json = { a: "say \"hi\"\n", cleared: "", extra: { beta: "on", nested: { x: 1 } } }
assertEq(encodeForm(params), "a=say%20%22hi%22%0A&cleared=&beta=on&nested[x]=1")
assertEq(encodeForm(null), "")
assertEq(encodeComponent("a b/č~"), "a%20b%2F%C4%8D~")
}
// ---- webhooks ----
let SECRET = "whsec_test"
let EVENT = "{\"id\":\"evt_1\",\"type\":\"checkout.session.completed\",\"created\":1700000000,\"account\":\"acct_1\",\"data\":{\"object\":{\"id\":\"cs_1\",\"metadata\":{\"saleId\":\"s1\"}}}}"
let NOW = 1700000100
test "a signed event is verified and read" {
let event = try verify(EVENT, signature(EVENT, SECRET, NOW - 10), SECRET, { now: NOW })
assertEq(event.id, "evt_1")
assertEq(event.type, "checkout.session.completed")
assertEq(event.account, "acct_1")
assertEq(metadata(event.data.object.metadata, "saleId"), "s1")
// The same from the body's bytes, as a route gets it.
let again = try verify(
Bytes.fromUtf8(EVENT),
signature(EVENT, SECRET, NOW - 10),
SECRET,
{ now: NOW },
)
assertEq(again.id, "evt_1")
}
test "a signature made by Node's crypto verifies" {
// c.createHmac("sha256", "whsec_test").update("1700000090." + EVENT).digest("hex")
let header = "t=1700000090,v1=55712e82ccc665fe735ce74716bdbb865294afa0148576756a34f43b7534a5bb"
assertEq(signature(EVENT, SECRET, 1700000090), header)
assert((try verify(EVENT, header, SECRET, { now: NOW }) else null) != null)
}
test "any of several signatures may match, as during a secret's rotation" {
let good = signature(EVENT, SECRET, NOW).split(",")[1]
let header = `t=${NOW},v1=${"0".repeat(64)},${good},v0=ignored`
assert((try verify(EVENT, header, SECRET, { now: NOW }) else null) != null)
}
test "what isn't Stripe's is refused" {
let header = signature(EVENT, SECRET, NOW)
let _, tampered = try verify(EVENT + " ", header, SECRET, { now: NOW })
assertEq(tampered, SignatureMismatch)
let _, wrongKey = try verify(EVENT, header, "whsec_other", { now: NOW })
assertEq(wrongKey, SignatureMismatch)
let _, missing = try verify(EVENT, "", SECRET, { now: NOW })
assertEq(missing, MissingSignature)
let _, malformed = try verify(EVENT, "garbage", SECRET, { now: NOW })
assertEq(malformed, MalformedSignature { header: "garbage" })
let _, noV1 = try verify(EVENT, `t=${NOW},v0=abc`, SECRET, { now: NOW })
assertEq(noV1, MalformedSignature { header: `t=${NOW},v0=abc` })
let _, badHex = try verify(EVENT, `t=${NOW},v1=zz`, SECRET, { now: NOW })
assertEq(badHex, MalformedSignature { header: `t=${NOW},v1=zz` })
}
test "an old event is refused as a replay, unless the check is off" {
let header = signature(EVENT, SECRET, NOW - 301)
let _, old = try verify(EVENT, header, SECRET, { now: NOW })
assertEq(old, TimestampOutOfTolerance { age: 301 })
assert((try verify(EVENT, header, SECRET, { now: NOW, tolerance: 0 }) else null) != null)
assert((try verify(EVENT, header, SECRET, { now: NOW, tolerance: 600 }) else null) != null)
}
test "a signed payload that isn't an event is refused" {
let payload = "{\"id\":1}"
let _, err = try verify(payload, signature(payload, SECRET, NOW), SECRET, { now: NOW })
match err {
MalformedPayload { message } => assert(message.includes("isn't")),
_ => assert(false, "a malformed payload"),
}
}
// ---- errors ----
fn body(s: string): json {
return try parse(s) else null
}
test "errors map to their types" {
let card = failureFrom(
402,
body(
"{\"error\":{\"type\":\"card_error\",\"code\":\"card_declined\",\"decline_code\":\"insufficient_funds\",\"message\":\"Declined\",\"param\":\"number\"}}",
),
"req_1",
)
match card {
CardError { failure } => {
assertEq(failure.decline_code, "insufficient_funds")
assertEq(failure.request_id, "req_1")
},
_ => assert(false, "a card error"),
}
assertEq(errorMessage(card), "Declined")
assertEq(failureFrom(401, null, null).kind, "AuthenticationFailed")
assertEq(failureFrom(403, null, null).kind, "PermissionDenied")
assertEq(failureFrom(404, null, null).kind, "InvalidRequest")
assertEq(failureFrom(400, null, null).kind, "InvalidRequest")
assertEq(failureFrom(429, null, null).kind, "RateLimited")
assertEq(failureFrom(500, null, null).kind, "ApiFailure")
assertEq(
failureFrom(400, body("{\"error\":{\"type\":\"idempotency_error\"}}"), null).kind,
"IdempotencyConflict",
)
assertEq(errorMessage(failureFrom(500, null, null)), "Stripe answered 500")
}
test "expandable fields" {
assertEq(idOf("cus_1"), "cus_1")
assertEq(idOf(body("{\"id\":\"cus_2\",\"email\":\"a@b.c\"}")), "cus_2")
assertEq(idOf(null), null)
}
test "a client's defaults" {
let c = client("sk_test")
assertEq(c.baseUrl, "https://api.stripe.com")
assertEq(c.maxRetries, 2)
assertEq(client("sk_test", { maxRetries: 0, stripeAccount: "acct_1" }).stripeAccount, "acct_1")
}

114
stripe/webhooks.weft Normal file
View file

@ -0,0 +1,114 @@
// Webhook events: checking that one really came from Stripe.
import { parse } from "std/json"
import { floor } from "std/math"
import { now } from "std/time"
import { constantTimeEqual, hmacSha256 } from "./lib/sha256"
/// An event Stripe sent. `data.object` is the object it is about: cast it with `as`, to the type of the event.
/// `account` is the connected account it happened on, for Connect endpoints.
export type Event = {
id: string,
type: string,
created: float,
livemode: boolean?,
api_version: string?,
account: string?,
pending_webhooks: float?,
data: { object: json, previous_attributes: json },
}
/// Why an event was refused. Answer any of them with a 400.
export type WebhookError =
| MissingSignature
| MalformedSignature { header: string }
| SignatureMismatch
| TimestampOutOfTolerance { age: float }
| MalformedPayload { message: string }
/// `tolerance` (300) is how old an event may be, in seconds, against replays; `0` turns the check off. `now` is the time in
/// seconds since the epoch (the clock's).
export type VerifyOptions = { tolerance?: float, now?: float }
fn bytesOf(payload: Bytes | string): Bytes {
return payload is string ? Bytes.fromUtf8(payload) : payload
}
fn signatureOver(payload: Bytes, secret: string, timestamp: float): Bytes {
return hmacSha256(Bytes.fromUtf8(secret), Bytes.fromUtf8(`${timestamp}.`).concat(payload))
}
/// Checks the `stripe-signature` header against the raw request body, and gives the event. Pass the body exactly as it
/// arrived: parsing and re-encoding it changes the bytes the signature covers.
///
/// ```weft
/// import { verify } from "@weft/stripe/webhooks"
///
/// let router = http.router().post("/stripe/webhook", async req => {
/// let event, err = try verify(req.body, req.headers.get("stripe-signature") ?? "", secret)
/// if err != null {
/// return http.status(400)
/// }
/// if event.type == "checkout.session.completed" {
/// let session = try event.data.object as Session
/// }
/// return http.json({ received: true })
/// })
/// ```
export fn verify(
payload: Bytes | string,
header: string,
secret: string,
options: VerifyOptions = {},
): Event ! WebhookError {
if header == "" {
fail MissingSignature
}
let mut timestamp: float? = null
let mut signatures: Bytes[] = []
for part of header.split(",") {
let at = part.indexOf("=")
let name = part.slice(0, at < 0 ? 0 : at).trim()
let value = part.slice(at + 1).trim()
if name == "t" {
timestamp = try float(value) else null
} else if name == "v1" {
let decoded = try Bytes.fromHex(value) else null
if decoded != null {
signatures.push(decoded)
}
}
}
if timestamp == null || signatures.length == 0 {
fail MalformedSignature { header }
}
let body = bytesOf(payload)
let expected = signatureOver(body, secret, timestamp)
// Every signature is compared, so the time taken doesn't say which one matched.
let mut matched = false
for signature of signatures {
matched = constantTimeEqual(expected, signature) || matched
}
if !matched {
fail SignatureMismatch
}
let tolerance = options.tolerance ?? 300
let current = options.now ?? floor(now().epochMs / 1000)
let age = current - timestamp
if tolerance > 0 && age > tolerance {
fail TimestampOutOfTolerance { age }
}
let parsed, bad = try parse(body.toUtf8())
if bad != null {
fail MalformedPayload { message: bad.message }
}
let event, wrong = try parsed as Event
if wrong != null {
fail MalformedPayload { message: `${wrong.path} isn't ${wrong.expected}` }
}
return event
}
/// A `stripe-signature` header for `payload`, for tests of your own webhook handler.
export fn signature(payload: Bytes | string, secret: string, timestamp: float): string {
return `t=${timestamp},v1=${signatureOver(bytesOf(payload), secret, timestamp).toHex()}`
}

11
stripe/weft.toml Normal file
View file

@ -0,0 +1,11 @@
[package]
name = "weft/stripe"
version = "0.1.0"
description = "A typed client for Stripe's API: customers, Checkout, payments, subscriptions, prices, products and webhooks"
license = "MIT OR Apache-2.0"
# A placeholder until the public repository exists (packages/README.md, Publishing for real).
repository = "https://git.weft-lang.dev/weft/packages"
hosts = ["api.stripe.com"]
exports = ["stripe", "customers", "checkout", "payment_intents", "subscriptions", "prices", "products", "refunds", "billing", "webhooks"]
main = "stripe"
include = ["*.weft", "lib", "README.md"]