Hi tribe! I'm trying to figure out how people think about and build APIs? 1️⃣ I design db schema first and add API endpoints that mostly reflect db schema. 2️⃣ I design my API schema and endpoints in my framework's code first. 3️⃣ I use a DSL/tool to define the schema and generate code. 4️⃣ Something else. If possible, please mention framework/tool and any quirks of your process 🙂

In general in mid to large applications, with relational DB, cache and message queue, the data model takes precedence. API endpoints are mostly there to enable easy ingress/egress (from an API consumer's point of view) and for authentication/authorization. Like in API, some data might be de-normalized for ease of use for consumer, but in DB it is normalized.

Thank you @wide-carpet-65149 What purpose does the MQ serve in applications you write? Is it primarily there for background tasks? And/or is your application event driven where the MQ serves as a transport for events?

Depending on the situation or need, both are possible. Broadly speaking background tasks or a means to tie loosely coupled systems. I generally keep HTTP handlers light, so that's where MQ appears first.

Offtopic: Any recommendations for books/resources to understand how to design/build APIs?

@limited-cartoon-72739 Public API Specs from some large places: https://apis.guru/

https://apisyouwonthate.com/ has book recommendations, articles etc. My personal process switches between 2 and 3 depending on which one is easier. It was surprising to see the washout 😅 If you're trying to get better at API design, the simplest way to think about it is to start with the user experience and walk backwards. Write down use cases and design APIs to make those use cases easier, your db schema would be a normalised form of this structure. Initially you'll run into design challenges with db schema and queries. You might need to change your APIs to find a middleground between UX and perf. But over time you'll get used to it and almost get it in your first attempt.

@dry-monkey-93718 I think that is a poor idea that I keep seeing people suggest. Consumers of an API are like users of a product. They do not dictate all the usage. You maintain the software, hopefully, a lot longer than users, who generally come and go and suggest a hundred things. Also, in most applications API is not what users' see directly, they see an app which consumes the API.

I generally design the API in terms of its responsibilities/operations with respect to it's environment first (users, upstream services etc). The DB schema usually comes second, since it's implementation detail, and at times you may need switch/augment your persistence layer to accomodate use cases. For example, we had a discovery service that we wrote using elasticsearch backend, but later we had to switch to a graph db. Also sometimes you may need to split/refactor your DB model down the road to optimise queries. Tying your API to your DB schema introduces needless coupling of your application interfaces to your dependencies.

I guess I am mainly dealing with and talking about APIs for internal use, for frontend apps. That is the common use case of APIs since most frontends (web or native) needs some storage/service/authentication API to talk to. In my mind these APIs are similar to RPCs. Just like functions get refactored, APIs also get refactored as application matures. The state of the application is shared through the API in ways that make most sense to the UX. The API itself is just an implementation detail like @helpful-mechanic-54614 you mentioned the DB schema is an implementation detail. There is no "tying" of API to DB schema though and I never understand what people mean by that. It is not like an API can do things outside what the API handlers allow. So there is a direct connection. In most applications I have dealt with, the API is structured way for two different systems to communicate (usually JavaScript/Python in my case). Python exists to hide the details of database, cache, etc. But the larger the application becomes, it is sometimes beneficial for JavaScript to directly deal with more of the data model. For instance, images being served would not be proxied through Python. They are stored in a storage (like S3) and URLs are part of the User entity. Similarly, there may be other cached data in some other part of the system which our frontends can directly pull. In these cases the data model is known to the frontend applications. Again, internal APIs. The way I understand, GraphQL is an example where APIs become even more like RPCs and consumer apps have full knowledge of data model.

I think that is a poor idea that I keep seeing people suggest.

umm, 😄

Consumers of an API are like users of a product. They do not dictate all the usage.

I think this is at the heart of the difference between the schools of thought. For me, users of the API and their needs come first. As with any product, the makers should not create bloat or cater to all wishes of every single user. PS: Typically the user of an API is the other team/frontend not the end user. In some cases, the user and the end users are the same (API products). There I'd say going API first is pretty much essential. Even when they're not, a lot of the times this has a tangible effect on end user's experience. For example, I often see frontend make multiple calls to fetch separate related pieces of information from different services/apps, this causes parts of the application to feel wonky or slow down/error out in weird ways sometimes @wide-carpet-65149 you might really like graphQL kind of an api layer.

@dry-monkey-93718 like I mentioned, if your product is the API, then the thought process should be different. Since you are selling literally the API. In my case, I have never done that. We sell frontends. Users do not care about what happens behind. I am not a GraphQL user but I totally understand where it comes from. The modern avatar of APIs is an incarnation of RPCs. GraphQL just makes the data model available to frontends (almost as if the entire API is an RPC) since frontends are powerful machines directly in the hands of the user. Thereby avoiding certain types of API design process altogether. For my kind of applications, APIs exist only because we have to make two applications talk over HTTP(S). Applications not managing requests well is totally different problem. Many developers tend to think JavaScript in the browser is not good enough and try to compensate at the API layer. To your point of "separate related" - if the entity model is such, then why not? Like user and user's picture. They are related but different services. It makes sense in my mind to merge in frotnend, not in backend or single API.

It makes sense in my mind to merge in frotnend, not in backend or single API.

Multiple calls, especially when done back to back (one call to fetch current user and then next to get related information), make your UI look slower. If you have an enterprise app, probably that won't matter much as the deal has happened at a much higher level than actual users. For consumer apps, it's very important.