Archive of posts with tag 'open source'
gpt-researcher →
GPT Researcher is an autonomous agent designed for comprehensive online research on a variety of tasks.
The agent can produce detailed, factual and unbiased research reports, with customization options for focusing on relevant resources, outlines, and lessons. Inspired by the recent Plan-and-Solve and RAG papers, GPT Researcher addresses issues of speed, determinism and reliability, offering a more stable performance and increased speed through parallelized agent work, as opposed to synchronous operations.
I’ve been tinkering around with this tool for doing deeper topic research. It uses the ChatGPT API and GPT-4o to conduct multi-phase agent-based research. Adds sources, citations,...
Ancient Myths and Open Source →
Sam Arbesman finds similarities between the oral storytelling of myth. With generation retellings and small, effective adaptations over time, ancient tales were made more Lindy:
Over time though, each ever-retold story was sanded into a gem of a tale that could last all those years, not necessarily because the written text was preserved—to be found in a clay pot in some desert cave millennia hence, or etched into a rock so it could not be forgotten—but because there was a community devoted to its recounting.
OpenStreetMap is Having a Moment →
My friend Joe Morrison wrote an excellent piece on the current state of corporate investment in OpenStreetMap:
The four companies in the inner circle — Facebook, Apple, Amazon, and Microsoft — have a combined market capitalization of over six trillion dollars.¹ In almost every other setting, they are mortal enemies fighting expensive digital wars of attrition. Yet they now find themselves eagerly investing in and collaborating on OSM at an unprecedented scale (more on the scale later).
The post references Jennings Anderson’s analysis of corporate contributions to the project (check out the
Weekend Reading: Software Builders, Scarcity, and Open Source Communities
👨💻 We Need More Software Builders, Not Just Users
On the announcement of Airtable’s latest round and $2.5b valuation (!), founder Howie Liu puts out a great piece on the latest round of changes in pursuit of their vision.
No matter how much technology has shaped our lives, the skills it takes to build software are still only available to a tiny fraction of people. When most of us face a problem that software can answer, we have to work around someone else’s...
Mermaid Diagrams →
Mermaid is a neat project to add a custom syntax layer in Markdown to support rendering diagrams like flowcharts and sequences.
Roam just recently added support for Mermaid in Roam graphs, which renders your diagram live as you edit the markup in a page. This is useful for adding visuals to pages, but I’ll have to mess around and learn its syntax to get comfortable.
Digital Gardeners →
The “digital garden” concept is gaining in popularity. I’ve seen a dozen of them recently, with groups like the Roam community taking to publishing their own open notebooks.
Maggie Appleton (an awesome, prolific member of the #roamcult) created this small library of resources for creating your own garden, along with several examples of others in the community. I still have the idea in my backlog of side projects to look at spinning up an open notebook like this.
Here’s Maggie’s illustration...
Weekend Reading: Beastie Boys, Links, and Screencasting
🎥 Beastie Boys Story
We watched this a couple nights ago. It’s hard to tell how objectively good it was, but I loved the heck out of it as a decades-long fan.
🔗 Linkrot
I’ll have to try out this tool that Tom built for checking links. When I’ve run those SEM tools that check old links, I get sad seeing how many are redirected, 404’d, or dead.
📹 Screencasting Technical Guide
This is an excellent walkthrough on how to make screencasts. I’ve done my own tinkering...
Weekend Reading: Tagging with Turf, Mars Panorama, and Kinds of Easy
🗺 turf-tagger
Bryan put together this neat little utility for merging point data with containing polygon attributes with spatial join queries. It uses Turf.js to do the geoprocess in the browser.
🚀 Mars Curiosity High-Res Panorama
Amazing photography of the Mars surface:
NASA’s Curiosity rover has captured its highest-resolution panorama yet of the Martian surface. Composed of more than 1,000 images taken during the 2019 Thanksgiving holiday and carefully assembled over the ensuing months, the composite contains 1.8 billion pixels of Martian landscape. The rover’s...
Why Hasn’t Open Source Software Disrupted Esri? →
This is a great post from my friend Joe Morrison assessing the state of the open source software movement in GIS against the biggest corporate competitive friction point, Esri.
Even as QGIS and comparable open source projects have invested in better branding, that doesn’t mitigate the ultimate killer in the quest to replace Esri at most large organizations: misaligned incentives. The typical IT department at a large company or government agency is defensive by nature due to the asymmetrical risk of adopting new technology; you can get fired for championing a new software that winds up...
Weekend Reading: Blot, Hand-Drawn Visualizations, and Megafire Detection
📝 Blot.im
Blot is a super-minimal open source blogging system based on plain text files in a folder. It supports markdown, Word docs, images, and HTML — just drag the files into the folder and it generates web pages. I love simple tools like this.
🖋 Handcrafted Visualization: Precision
An interesting post from Robert Simmon from Planet. These examples of visualizations and graphics of physical phenomena (maps, cloud diagrams, drawings of insects, planetary motion charts) were all hand-drawn, in an era where specialized photography and sensing weren’t always options.
...
mapwith.ai →
In a conversation yesterday I learned about this project called RapiD, led by Facebook to use computer vision technology to detect features for mapping in OpenStreetMap. They’re working on a fork of the iD editor that does assistive feature detection to present an editor with generated geometries to add to the map. I messed around with it in the test areas they’re supporting so far and it’s a clever combination of computer-assisted detection and human-based mapping. This shows some promise to enhance OSM contributor technology and lower the barrier to entry for new editors.
bookish.tech →
This is a handy tool for searching book metadata, built by Tom MacWright. It’s a “universal converter” for book IDs in various services, like Goodreads, OpenLibrary, and WorldCat.
It’s open source and also has an API. I’ll have to see about factoring these other IDs into my own Library reading log.
Search the Archives
Since I’ve been posting here so frequently, it’s gotten challenging to scroll through the archive to find links to things I wrote about before. Last night I worked on implementing a simple site search page that searches the title, text, and tags of posts to find relevant content. This is a short post on how I put that together.
I use Jekyll to manage the site content and generation, with all of my posts written as markdown files with some custom front-matter to handle things like tagging, search-friendliness,...
Weekend Reading: tracejson, Euclid, and Designing at Scale
🛰 tracejson
An extension to the GeoJSON format for storing GPS track data, including timestamps. GPX has been long in the tooth for a long time, but it works and is supported by everything. This approach could have some legs if application developers are interested in a new, more efficient, way of doing things. I know I’d like to explore it for Fulcrum’s GPS-based video capability. Right now we do GPX and our own basic JSON format for storing the geo and elapsed time data to match up video frames with location. This could...
Weekend Reading: Terrain Mesh, Designing on a Deadline, and Bookshelves
🏔 MARTINI: Real-Time RTIN Terrain Mesh
Some cool work from Vladimir Agafonkin on a library for RTIN mesh generation, with an interactive notebook to experiment with it on Observable:
An RTIN mesh consists of only right-angle triangles, which makes it less precise than Delaunay-based TIN meshes, requiring more triangles to approximate the same surface. But RTIN has two significant advantages:
- The algorithm generates a hierarchy of all approximations of varying precisions — after running it once, you can quickly retrieve a mesh...
MapSCII →
I love that people out in the open source world still build things like this. It’s an xterm-compatible renderer for accessing interactive map from a terminal console.
I even tried it from a console on an EC2 instance I’ve got. A quick telnet command gets you connected, a and z keys to zoom, arrow keys to pan:
telnet mapscii.me
Funnily enough, I could even see this being useful if you needed to reference a map while on a command line on a remote server. Mostly it’s a clever toy,...
Open Source Game Clones →
I don’t play games at all anymore these days, but I ran across this repository of open source clones of classic PC games.
Some notable ones from the list that we used to play:
- 0 AD — Age of Empires
- C-Evo — Civilization II
- Interstate Outlaws — Interstate ‘76
- Jedi Outcast Linux — Jedi Knight II: Jedi Outcast
- OpenRCT — Rollercoaster Tycoon
- OpenSkyscraper — SimTower
The world of open source software (and GitHub, specifically) really is amazing. Kids out there the age I was when I played these can now go online...
Assisted Tasks for OpenStreetMap →
The Humanitarian OpenStreetMap Team has been working on an experimental version of the Tasking Manager tool that incorporates deep learning-assisted mapping projects.
The OSM community has long been (and still largely is) averse to machine-based mapping, as it’s counter to the founding ethos of the project being “created by mappers, where they live.” But if the project is to survive and still see adoption and usage in commercial applications, there has to be effort to improve the depth of coverage and refresh rate to stay competitive with the commercial providers like...
Discovering QGIS
This week we’ve had Kurt Menke in the office (of Bird’s Eye View GIS) providing a guided training workshop for QGIS, the canonical open source GIS suite.
#qgis workshop selfie @spatialnetworks ... a great group hard at work! pic.twitter.com/fp1jNsQMFL
— Kurt Menke (@geomenke) May 28, 2019
It’s been a great first two days covering a wide range of topics from his book titled Discovering QGIS 3.
The team attending the workshop is a diverse group...
Contours in OpenDroneMap →
One of my favorite open source projects ongoing right now is OpenDroneMap. I haven’t gotten to use it much recently, but followed development over the last couple years. Outside of some loose testing a while ago, I haven’t flown my Mavic for any imagery collection since. I need to go out to the waterfront nearby and fly some new data so I can kick the tires some more on ODM’s latest stuff.
Piero just announced completion of contour support in WebODM, which is the web front-end to the...
A Growing Geo Community: Takeaways from FOSS4G NA 2019 →
I had a great time last week out in San Diego for this year’s FOSS4G-NA. This post I wrote for the company blog runs down general conference takeaways, plus a wavetops overview with best-of sessions I attended. It was a fantastic event, as always. The content was great, but the real win is getting to hang out with friends old and new and stay in touch with what everyone’s working on:
The key to getting the most out of a FOSS4G is the people. I haven’t attended one since 2013, but each one I’ve been to...
FOSS4G North America 2019
Next week Joe and I will be out in San Diego for FOSS4G-NA 2019. This’ll be my first one since I think 2012. There’s always an excellent turnout and strong base of good folks to catch up with. This year they’ve put together a B2B and Government Theme day to kick it off, which to my knowledge is a new thing for an event typically focused on the eponymous free, open source, and community-driven projects.
I thumbed through the agenda to pick out some topics I’m interested in catching this year:
- ...
Sponsoring QGIS →
We recently began a corporate sponsorship for the QGIS project, the canonical open source desktop GIS. I got back into doing some casual cartography work using QGIS back in December after a years-long hiatus of using it much (I don’t get to do much actual work with data these days). Bill wrote up a quick post about how we rely on it every day in our work:
As a Mac shop, the availability of a sophisticated, high-quality, powerful desktop GIS that runs natively on MacOS is important to us. QGIS fits that bill and more. With its...
Weekend Reading: Calculator, SaaS Metrics, and System Shock
💻 Open Sourcing Windows Calculator
Seems silly, but this kind of thing is great for the open source movement. There’s still an enormous amount of tech out there built at big companies that creates little competitive or legal risk by being open. Non-core tools and libraries (meaning not core to the business differentiation) are perfect candidates to be open to the community. Check it on GitHub.
📊 The Metrics Every SaaS Company Should Be Tracking
An Inside Intercom interview with investor...
Desktop GIS — The Evergreen Topic →
A good post from my colleague Bill Dollins about the state of the desktop GIS:
I pretty much watch Esri from a distance these days but, even from my perch, I can’t see any workflow that doesn’t involve desktop software at some point. It’s simply still crucial to the editing and publishing of maps and other geospatial content.
That’s even true outside of the Esri world. My team lives in QGIS. As of this writing, QGIS is at version 3.4.4 and it’s hard for me to make a case that I...
Weekend Reading: Fulcrum in Santa Barbara, Point Clouds, Building Footprints
👨🏽🚒 Santa Barbara County Evac with Fulcrum Community
Our friends over at the Santa Barbara County Sheriff have been using a deployment of Fulcrum Community over the last month to log and track evacuations for flooding and debris flow risk throughout the county. They’ve deployed over 100 volunteers so far to go door-to-door and help residents evacuate safely. In their initial pilot they visited 1,500 residents. With this platform the County can monitor progress in real-time and maximize their resources to the areas that need the most attention.
“This...
Weekend Reading: Shanghai, Basecamp, and DocuSaurus
🇨🇳 195-Gigapixel Photo of Shanghai
Shot from the Oriental Pearl Tower, the picture shows enormous levels of detail composited from 8,700 source photos. Imagine this capability available commercially from microsatellite platforms. Seems like an inevitability.
🏕 How Basecamp Runs its Business
I, like many, have admired Basecamp for a long time in how they run things, particularly Ryan Singer’s work on product design. This talk largely talks about how they build product and work as an organized team.
📄 Docusaurus
This is an open source framework for building documentation sites, built with React. We’re...
Fulcrum Desktop
A frequent desire for Fulcrum customers is to maintain locally a version of the data they collect with our platform, in their database system of choice. With our export tool, it’s simple to pull out extracts in formats like CSV, shapefile, SQLite, and even PostGIS or GeoPackage. What this doesn’t allow, though, is an automatable way to keep a local version of data on your own server. You’d have to extract data manually on some schedule and append new stuff to existing tables you’ve already got.
A while back we built and...
Kindle Highlights
I started making this tool a long time back to extract highlighted excerpts from Kindle books. This predated the cool support for this that Goodreads has now, but I still would like to spend some time getting back to this little side project.
Eric Farkas has another tool that looks like it does this, as well, so that’s worth checking out as a possible replacement. What I really want is my own private archive of the data, not really my own custom extraction tool. The gem I was using for mine might’ve been the same one,...
OpenDroneMap
Since I got the Mavic last year, I haven’t had many opportunities to do mapping with it. I’ve put together a few experimental flights to play with DroneDeploy and our Fulcrum extension, but outside of that I’ve mostly done photography and video stuff.
OpenDroneMap came on a scene a couple years ago as a toolkit for processing drone imagery. I’ve been following it loosely through the Twittersphere since. Most of my image processing has been done with DroneDeploy, since we’d been working with them on some integration between...
Jupyter, Mathematica, and the Future of the Research Paper →
Paul Romer has an interesting take on this piece from the Atlantic:
Jupyter rewards transparency; Mathematica rationalizes secrecy. Jupyter encourages individual integrity; Mathematica lets individuals hide behind corporate evasion. Jupyter exemplifies the social systems that emerged from the Scientific Revolution and the Enlightenment, systems that make it possible for people to cooperate by committing to objective truth; Mathematica exemplifies the horde of new Vandals whose pursuit of private gain threatens a far greater public loss–the collapse of social systems that took centuries to build.
The original article is a great read, touching...
JekyllConf →
JekyllConf is coming up in May, a conference to talk about the latest with Jekyll, the static site generator that lets you publish websites without heavy needs for hosting and databases.
This isn’t a typical conference, either. It’s all online, streaming live via Google Hangouts on Air. Last year there were great presentations from people worldwide on things they’re doing with the framework.
We converted our website some time back to use Jekyll for the entire thing. We started on Wordpress, which at the time made anything that wasn’t a blog post annoying to deal with....
speedtest-cli →
If you’re a nerd about checking your download and upload bandwidth when things seem to be getting slow, you’ve certainly used Speedtest.net to find out your raw internet speed. If you’re an even bigger nerd, you’ll love this command line tool for running quick tests from the shell. The Speedtest website is so covered with nonsense these days, I wonder how much of my bandwidth is being taken to display Flash ads and run Javascript trackers.
It’s not a 100% accurate tool, but it gets me what I need.
AlaSQL →
A neat find for doing SQL queries in the browser. Open source and easy compatibility with tons of data formats.
Bringing Geographic Data Into the Open with OpenStreetMap
This is an essay I wrote that was published in the OpenForum Academy’s “Thoughts on Open Innovation” book in early summer 2013. Shane Coughlan invited me to contribute on open innovation in geographic data, so I wrote this piece on OpenStreetMap and its implications for community-building, citizen engagement, and transparency in mapping. Enjoy.
With the growth of the open data movement, governments and data publishers are looking to enhance citizen participation. OpenStreetMap, the wiki of world maps, is an exemplary model for how to build community and engagement...
Terra
Inspired by a couple of others, I released a micro project of mine called Terra, to provide a fast way to run several geospatial tools on your computer.
Because I work with a variety of GIS datasets, I end up writing lots of scripts and small automation utilities to manipulate, convert, and merge data, in tons of different formats. Working with geo data at scale like this challenges the non-software developer to get comfortable with basic scripting and programming. I’ve learned a ton in the last couple years about Unix environments,...
Creating New Contributors to OpenStreetMap
I wrote a blog post last week about the first few months of usage of Pushpin, the mobile app we built for editing OpenStreetMap data.
As I mentioned in the post, I’m fascinated and excited by how many brand new OpenStreetMap users we’re creating, and how many who never edited before are taking an interest in making contributions. This has been an historic problem for the OpenStreetMap project for years now: How do you convince a casually-interested person to invest the time to learn how to contribute themselves?
There are two...
Symbola
If you’ve ever done much involving symbol sets in mapping (especially web mapping), you know about the nightmare of managing 700 separate PNG files, with different, duplicate versions for slightly different size variations and colors. Even with a small set of a dozen symbols, if you want 3 sizes and 5 colors for each, you’re looking at 180 distinct PNG marker symbols to keep track of. Ugh. SVG format simplifies this in certain ways, but isn’t as universally supported or easy to work with as simple GIFs or PNGs.
With TileMill, I’ve wanted to use marker symbolizers frequently in...
WhereCampTB
My talk from Ignite Spatial at WhereCampTB, talking about the OSM Tampa Bay meetup group. Check out the slides in better detail here.
It was a fun event a couple weeks ago — great participation from folks in all sorts of industries involved in mapping or using GIS tools.
OSM in Commercial Products
OpenStreetMap has become an undeniably powerful open data resource for industry to start taking advantage of. I gave this talk at State of the Map 2011 in Denver to show some of the things our company is doing leveraging OSM data.
WhereCampDC
We just returned from a fantastic weekend up in DC - first at the Ignite Spatial event on Friday night, then the WhereCampDC unconference on Saturday. Being the first event of it’s kind that I’ve attended (with the “barcamp” unconference session format), I thought I’d write up some thoughts and impressions from an amazing 2-day trip.
Ignite Spatial
This was also my first experience hearing talks in the ignite format—20 slides, 15 seconds each, 5 minutes. A fantastic format to break people out of the habit of simply reading their slides off a screen. Held at Grosvenor Auditorium...