charmaineValSearch
Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data – all from the browser, and deployed in milliseconds.
Viewing readonly version of main branch: v41View latest version
A powerful search engine for Val Town projects that enables searching across code files, project descriptions, and user data.
- Full-text search across all project files and content
- Enhanced project descriptions generated using AI analysis of README files and code
- Multi-type search supporting files, projects, users, and documentation
- Real-time typeahead suggestions for usernames and project names
- Newsletter curation tool for finding active projects
- JSON API access for programmatic usage
The system now automatically generates concise, one-line descriptions for projects by analyzing:
- Existing project descriptions from Val Town API
- README files (README.md, README.txt, etc.)
- Main code files (index.ts, main.js, app.tsx, etc.)
- File structure and naming for additional context
When importing or updating projects, the system:
- Checks if the project has a good existing description
- Looks for README files in the project root
- Analyzes main code files if no README is found
- Uses OpenAI to generate a concise summary (under 100 characters)
- Stores the enhanced description in the database
You can manually trigger description enhancement for existing projects:
GET /enhance-descriptions?limit=50
This endpoint will:
- Find projects with poor or missing descriptions
- Generate AI summaries for up to the specified limit
- Return statistics about the enhancement process
Projects are considered to need enhanced descriptions if they:
- Have no description
- Have empty descriptions
- Have descriptions shorter than 10 characters
- Contain placeholder text like "todo" or "description"
GET /?q=searchterm
- Main search interfaceGET /?q=searchterm&format=json
- JSON API accessGET /?q=searchterm&type=files|projects|users|docs
- Filter by result type
GET /typeahead?q=partial
- Get typeahead suggestions
GET /newsletter?days=30
- Find active projects for newsletterGET /newsletter?days=30&format=json
- JSON format
GET /enhance-descriptions?limit=10
- Enhance project descriptions
The AI generates concise, action-oriented descriptions:
- "Creates interactive data visualizations from CSV files"
- "Provides real-time weather data API with caching"
- "Generates QR codes with custom styling options"
- "Converts Markdown to HTML with syntax highlighting"
The system requires:
- SQLite database for storing project and file data
- OpenAI API access for description generation
- Val Town SDK for data import
Environment variables:
- OpenAI API key (automatically available in Val Town environment)
The system uses three main tables:
projects
- Project metadata and enhanced descriptionsfiles
- File content for search indexingusers
- User profile information
To contribute to the project description enhancement:
- Modify the prompt in
enhance-descriptions.ts
to improve summary quality - Adjust the enhancement criteria in
db.ts
- Add new file type support for analysis
- Improve the README detection logic
- Description enhancement processes 10-50 projects per request
- Includes rate limiting to avoid API throttling
- Uses efficient database queries with pagination
- Caches user data to avoid redundant API calls