Feedscout

Appearance

Customize URL Resolution ​

Feedscout resolves discovered URLs before validation to ensure consistent results. You can provide a custom resolution function to change this behavior.

Default Behavior ​

By default, Feedscout resolves relative URLs against the base URL:

typescript
// Base URL: https://example.com/blog/
// Discovered: /feed.xml
// Resolved: https://example.com/feed.xml

// Base URL: https://example.com/blog/
// Discovered: ../rss
// Resolved: https://example.com/rss

Custom Resolution ​

Provide a resolveUrlFn to customize URL resolution:

typescript
import type { DiscoverResolveUrlFn } from 'feedscout'

const customResolve: DiscoverResolveUrlFn = (url, baseUrl) => {
  // Resolve relative URLs
  const resolved = new URL(url, baseUrl).href

  // Remove tracking parameters
  const parsed = new URL(resolved)
  parsed.searchParams.delete('utm_source')
  parsed.searchParams.delete('utm_medium')
  parsed.searchParams.delete('utm_campaign')

  return parsed.href
}

const feeds = await discoverFeeds(url, {
  methods: ['html', 'guess'],
  resolveUrlFn: customResolve,
})

// Also works with discoverHubs
const hubs = await discoverHubs(url, {
  resolveUrlFn: customResolve,
})

Interface ​

typescript
type DiscoverResolveUrlFn = (url: string, baseUrl: string | undefined) => string | undefined

Return undefined when the URL cannot be resolved. Feedscout then keeps the URL as discovered. The same happens when the function throws, and the error is reported to onError, so a malformed href on a page does not stop discovery.

Use Cases ​

Removing Query Parameters ​

Strip unnecessary query parameters:

typescript
const resolveUrl: DiscoverResolveUrlFn = (url, baseUrl) => {
  const resolved = new URL(url, baseUrl)

  // Keep only essential parameters
  const essentialParams = ['format', 'type']
  const params = new URLSearchParams()

  for (const key of essentialParams) {
    const value = resolved.searchParams.get(key)
    if (value) params.set(key, value)
  }

  resolved.search = params.toString()
  return resolved.href
}

Forcing HTTPS ​

Upgrade HTTP URLs to HTTPS:

typescript
const resolveUrl: DiscoverResolveUrlFn = (url, baseUrl) => {
  const resolved = new URL(url, baseUrl)
  resolved.protocol = 'https:'
  return resolved.href
}

Removing Trailing Slashes ​

Remove trailing slashes from paths:

typescript
const resolveUrl: DiscoverResolveUrlFn = (url, baseUrl) => {
  const resolved = new URL(url, baseUrl)
  resolved.pathname = resolved.pathname.replace(/\/+$/, '')
  return resolved.href
}

Custom Domain Handling ​

Rewrite URLs for specific domains:

typescript
const resolveUrl: DiscoverResolveUrlFn = (url, baseUrl) => {
  const resolved = new URL(url, baseUrl)

  // Use feeds subdomain for specific sites
  if (resolved.hostname === 'example.com') {
    resolved.hostname = 'feeds.example.com'
  }

  return resolved.href
}

Combining with Other Options ​

URL resolution works with discoverFeeds, discoverBlogrolls, discoverFavicons, and discoverHubs:

typescript
const feeds = await discoverFeeds(url, {
  methods: ['html', 'guess'],
  resolveUrlFn: customResolve,
  extractFn: customExtractor,
  concurrency: 3,
})

const hubs = await discoverHubs(url, {
  methods: ['headers', 'html'],
  resolveUrlFn: customResolve,
})