Can you explain a bit further how do you see that working? I use CM because I want to control the app; I want to tell it where it should store its logs (so I can process them), where it should store its data (so I can back it up), which ports it should use (so I can point my reverse proxy at it), etc.
If the app just asks for stuff, where would the management part come it? And why have a CM at all? Granting resources is what the OS itself does; the application can just call tmpfile().
A traditional CM tool in this new paradigm (which i'm coining App Management before somebody steals the idea) would probably become more of a configuration fulfiller (CF) in this context. It fulfills the configuration requirements that the app dictates.
The CF could modify its behavior to fit a given platform. If you're on Windows, it will do operation X in one way, and on Linux a different way. You could describe to the CF the information it needs to fulfill requests - where are the big data volumes, where are the credentials, how should it connect remote resources, etc. When the app asks for something, the CF will have just enough information to figure out how to provide it for that case.
The app can label resources however it wants. For example, if it called tmpfile("my_logfile_access"), a CF could return it a temp file, and record in some datastore that application X asked for a logfile 'my_logfile_access' and that it was provided. If you want to process logs, ask the CF where the 'my_logfile_access' file/directory is. If you abstract away the need to tag it and just have a function called logfiles() or something, you wouldn't even need to know this tag - your app could just write logs with some tag metadata, so when you go to process the logs, you figure out what it is after you've opened the logs.
Same for data, and ports, etc. Why should I have a Vagrantfile that maps ports? The app knows what ports it wants to offer. And why use numerical ports? We've had /etc/services for like 30 years and yet nobody seems to realize you can use a name instead of a port number and stop worrying about manually mapping numbers around. Honestly, so much of modern devops stuff is backwards.
Your ideas resonate a lot with dependency injection [1] to me. However, I don't see how such an "App manager" would be able to setup, for instance, an nginx server with multiple virtual hosts.
Some applications need entire configuration languages because they are so flexible that writing some ini/json/xml quickly becomes too cumbersome. How would your system deal with those problems?
Finally, your proposed solution seems to suffer from having to adapt all applications to be able to manage them. This is impractical to say the least, and if you had the resources to take that adventure... I think you'd be better off turning to something like Nix
This is a dumb example, but illustrates one way this could happen. The actual CM in the background that the App_Manager library connects to would take these config directives and communicate with the other software (nginx) in order to set it up how the app above has specified.
Ideally the above would not include an Nginx-specific configuration, but one that was more abstract, to allow any HTTP service to provide for the needs of the application. That's probably wishful thinking. But on the other hand, there are probably millions of examples of an application needing an HTTP server in extremely similar ways, so maybe service-specific configuration specifications aren't totally insane.
> Some applications need entire configuration languages because they are so flexible that writing some ini/json/xml quickly becomes too cumbersome. How would your system deal with those problems?
You can abstract away almost all of the details of an automated system. They're just not built today with this in mind. Systems today are built as isolated islands of complexity and depend on humans to teach them how to shake hands. This is largely not necessary, but it is with how people are currently managing systems.
> Finally, your proposed solution seems to suffer from having to adapt all applications to be able to manage them
In the same way that networked applications "suffer" from adapting a standard protocol.
Another example:
#!/usr/bin/perl
use DBI;
$dsn = "dbi:SQLite:dbname=$dbfile";
$dbh = DBI->connect($dsn, $user, $password);
$sql = 'CREATE TABLE people (id INTEGER PRIMARY KEY, fname VARCHAR(100), lname VARCHAR(100), email VARCHAR(100) UNIQUE NOT NULL, password VARCHAR(20))';
$dbh->do($sql);
But this looks complicated; I guess asking apps to suffer under the concept of a standard interface and language will never catch on.
When I apt-get install most applications, I don't need to tell them where they should put their logs, they just the mkdir('/var/log/...') API. I don't see us gaining much by adding another man-in-the-middle to translate a logfiles(X) call to mkdir('/var/log/app')/open('/var/log/app/'+'/var/log/'+X).
The reason I use a CM is mostly to configure the system itself (such as - which apps I want installed in the first place) and to define application-specific configurations, which a configuration fulfiller would have no way of knowing how to do.
The app knows what ports it wants to offer.
No, the app knows what services it wants to offer; ports are shared resources, so they can't be defined by each app.
We've had /etc/services for like 30 years and yet nobody seems to realize you can use a name instead of a port number and stop worrying about manually mapping numbers around.
On the contrary; /etc/services is part of the old paradigm, in which each application got a static port that you had to manually configure, even if you used a text file to give it a label. In my systems, the CM takes care of that, by allocating any free port and taking of the bookkeeping to prevent double use.
That is the polar opposite of what I'm suggesting. The FHS is designed to tell every application where everything will be. My idea is to have every application tell the system where everything will be.
Part of the problem is systems having their own standard. What I'm suggesting would be a standard way of an application defining its own standard, which other applications or systems could interrogate and learn dynamically. It would effectively end a majority of platform-dependence, other than the platform-specific interfaces we unfortunately have to live with.
> No, the app knows what services it wants to offer; ports are shared resources, so they can't be defined by each app.
> On the contrary; /etc/services is part of the old paradigm, in which each application got a static port that you had to manually configure, even if you used a text file to give it a label
You are missing my point. The point is to get the application to dynamically pass a service which it needs or wants to share.
Ports are shared resources, but that does not mean they can't change. They are largely disposable, arbitrary numbers assigned to higher level concepts. We assign them statically because you can't easily bind one port to more than one application, and due to the inherent limitations of most protocols.
There is no reason I should need to tell Docker to "map internal port 80 to external port 18080". I should only have to tell it "I'm running http, please tell whoever connects to you (Docker) that I'm running http, and give them a port to access my http service on".
I don't mean to be offensive, but I'm not sure you've necessarily delved deep enough into the space of config management to understand that what you're talking about falls into three sort of areas: 1) irreducible complexity as a fact of life when the management mechanism must adapt to all possible use cases, 2) solved problems, where the means and best practices for doing what you're asking for is already there, and 3) asking for things that don't really make sense.
How so?
> The CF could modify its behavior to fit a given platform. If you're on Windows, it will do operation X in one way, and on Linux a different way.
This is exactly what most CM tools do. Chef is my specialty and in Chef, anything that makes sense to do on multiple platforms will, if done correctly, present a unified interface. Resource providers and community cookbooks abstract the OS-specific stuff from you, the CM administrator, so you can just include high level concepts. When done well, it's as simple as describing the end configuration for an app (generally and for the specific instance you're spinning up) and off it goes.
> If you abstract away the need to tag it and just have a function called logfiles() or something, you wouldn't even need to know this tag - your app could just write logs with some tag metadata, so when you go to process the logs, you figure out what it is after you've opened the logs.
If you look inside of any decently written application, you'll find that they use a centralized logging facility of some kind, which then demarcates the border between application and the running system. The app will provide a means of configuring its logging output to some extent, and then you're responsible for where it goes on the other side. Logs on disk, Event Viewer on windows, journald, or being sucked up into a logging pipeline like ELK, it's really up to the particular use case. Note that already, we're dealing with some pretty seriously different outputs; obviously this needs to be configured somewhere!
> Same for data, and ports, etc. Why should I have a Vagrantfile that maps ports? The app knows what ports it wants to offer.
Maybe the app does, but how should Vagrant / Docker / etc. know about this? The app could supply a Vagrantfile and a Dockerfile, but how does the app know what port I want open in reality into its own little port space? I may be doing something weird and not want the defaults for any number of reasons.
> We've had /etc/services for like 30 years and yet nobody seems to realize you can use a name instead of a port number and stop worrying about manually mapping numbers around.
/etc/services was almost never something you could rely on outside of the major protocols and if anything I'd prefer to just see it stop defining anything above 1024. It's kind of silly to think that there would only be ~64k different services in the world, now, but I suppose in the 70s it was more excusable.
> Honestly, so much of modern devops stuff is backwards.
There's backwardness, but I think some of it is in the way you're thinking about it. I don't mean to be insulting, but a really well managed CM environment absolutely does provide the majority of what you're asking for. It's not all out-of-the-box, maybe it could be, but that's sort of a pie in the sky way of thinking - "gee, what if all the app developers actually coordinated on something!" - yeah, might could work if everyone agreed to get over their own vision/ego and implement a common thing, but that's both unrealistic and also potentially stifling of innovation.
Devops is the job of surfing on top of the chaos and bringing as much order to it as you can so that when you inevitably burn out and leave the company, most of the good work you did has more momentum and inertia than it used to before we implemented common mechanisms of doing automation. Another Chef or Ansible devops guy can slot into your place, figure out the theme, and keep it going - even if it's not how they would have architected it from scratch. There's a hell of a lot of value in that.
If the app just asks for stuff, where would the management part come it? And why have a CM at all? Granting resources is what the OS itself does; the application can just call tmpfile().