Skip to content

Turn any website to API by several clicks (serverless and support SPA!)

License

Notifications You must be signed in to change notification settings

t9tio/cloudquery

Folders and files

NameName
Last commit message
Last commit date

Latest commit

01bd898 · Jul 24, 2020

History

36 Commits
Jul 25, 2019
Mar 26, 2019
Aug 1, 2019
Jan 23, 2019
Mar 25, 2019
Mar 25, 2019
Mar 25, 2019
Mar 25, 2019
Mar 25, 2019
Mar 25, 2019
Jul 23, 2020
Aug 1, 2019
Aug 1, 2019
Apr 3, 2019

Repository files navigation

CloudQuery

Join us

Turn any website to API by several clicks.

Try online: https://cloudquery.t9t.io

API

Query params:

  • url: url of the webpage you want to fetch
  • selectors: css selectors describing the elements interests you

Returns:

  • innerText: element innerText
  • href: element href

Run locally

# install dependencies
npm i
# start backend server(for auto restart when code change)
npm run sb
# start frontend server(for auto restart when code change)
npm run sf

# visit http://localhost:1234

How to deploy your own CloudQuery to AWS lambda

1. Configuration

  • update profile(AWS Credential Profiles) in up.json to use your own aws account
  • update rateLimit in config.json to set your own rate limit (Default: 5 request/hour)

2. Deploy

# deploy to aws lambda
up

# see the URL of your CloudQuery
up url

# deploy production version(to cloudquery.t9t.io)
up deploy production

Thanks

  • up for deploying serverless API to AWS with ease
  • serverless-chrome for running chrome on AWS lambda
  • finder for making it easy to select elements on webpage