| [11:53:40] | * zombiebeard has joined #aegir |
| [11:53:53] | * zombiebeard has quit (Client Quit) |
| [13:01:33] | * roycroft has quit (Ping timeout: 240 seconds) |
| [13:01:41] | * roycroft has joined #aegir |
| [16:47:39] | * realityloop has joined #aegir |
| [17:22:57] | * realityloop has quit (Quit: Leaving..) |
| [18:03:24] | * ybabel has joined #aegir |
| [18:14:31] | * reaper013 has joined #aegir |
| [21:58:31] | * ybabel has quit (Quit: ybabel) |
| [00:03:04] | * shaneonabike1 has joined #aegir |
| [00:33:52] | * jeremyr has joined #aegir |
| [00:55:06] | * jeremyr has quit (Quit: Leaving.) |
| [00:56:31] | * ybabel has joined #aegir |
| [01:16:01] | * ybabel has quit (Ping timeout: 248 seconds) |
| [02:56:46] | * reaper013 has quit (Quit: Page closed) |
| [04:57:48] | * memtkmcc has joined #aegir |
| [05:02:51] | <ergonlogic[m]> | helmo42 helmo colan memtkmcc jonpugh_ bgm Scrum time? Am I missing anyone? |
| [05:03:04] | <memtkmcc> | hi |
| [05:03:11] | <helmo42[m]> | Hi |
| [05:03:11] | <hefring> | privet |
| [05:03:52] | <ergonlogic[m]> | ok the core@ list, we'd planned to talk about the re-architecture proposals |
| [05:04:08] | <ergonlogic[m]> | any objections to proceeding along those lines? |
| [05:06:05] | <ergonlogic[m]> | For reference, here's the proposal that Colan and I wrote up last week: https://gitlab.com/aegir/aegir/wikis/architecture |
| [05:07:57] | <ergonlogic[m]> | @memtkmcc helmo42 I think it'll just be the three of us today. But you both expressed some concerns via email... |
| [05:08:24] | <bgm> | I'm reading :) |
| [05:08:33] | <ergonlogic[m]> | Hi Mathieu! |
| [05:10:08] | <memtkmcc> | The drush and drupal core timeline puts quite a pressure on us, so we may need to talk both about grand new AegirNG in this context while still trying to figure out if we can do things in phases, so both core dev work and users can continue upgrades in-place? |
| [05:11:16] | <helmo42[m]> | I have the feeling that NG is beyond the 6 month scope and that with modifications to provision we can move it closer to the goal of NG |
| [05:11:42] | <memtkmcc> | same |
| [05:11:53] | <ergonlogic[m]> | Right. I'm not proposing an either/or here. Jon's work is addressing the Drush issue head-on. |
| [05:12:08] | <ergonlogic[m]> | and that should probably become Aegir 4 |
| [05:12:28] | <ergonlogic[m]> | note that I think it's a pretty significant amount of work too, though |
| [05:12:59] | <ergonlogic[m]> | considering how much we depend on the drush API throughout Provision, and in plenty of Hosting components too |
| [05:14:27] | <ergonlogic[m]> | also, while the NG proposal is radically different in implementation, it keeps much of the overall architecture and terminology |
| [05:14:46] | <ergonlogic[m]> | In Aegir 3, we implement our own queues, and handle all the config mgmt ourselves |
| [05:15:07] | <ergonlogic[m]> | in NG, we propose to use Celery and Ansible to replace that |
| [05:15:27] | <ergonlogic[m]> | which greatly simplifies what we have to do oiurselves |
| [05:16:32] | <helmo42[m]> | getting 4.x all the way is definately quite some work ... one thing we have to keep reminding ourselvef of is "In which context is this?" |
| [05:16:48] | <ergonlogic[m]> | the time-line I'm looking at is (hopefully) a working prototype by the end of next week, and an minimal viable product by January |
| [05:17:00] | <ergonlogic[m]> | right |
| [05:17:08] | <ergonlogic[m]> | that'll goaway in NG |
| [05:17:35] | <ergonlogic[m]> | the Drupal8 db become the canonical source |
| [05:18:12] | <ergonlogic[m]> | we then marshal all the necessary context for a given task, and pass it all in |
| [05:18:30] | <ergonlogic[m]> | rather than depend on the backend having that data somewhere discoverable |
| [05:19:02] | <ergonlogic[m]> | the only exception that I'd like to see there being that secrets are handled mostly on the backend |
| [05:19:11] | <ergonlogic[m]> | probably via Ansible Vault for now |
| [05:20:45] | <ergonlogic[m]> | so, for example, if we provision a database, we store the credentials as variables that can be access by subsequent tasks that may need them |
| [05:21:03] | <memtkmcc> | dropping drush is going to generate a ton of work, no matter how we slice it, and will push us into NG scope anyway, but my suggestion is to try to identify and prioritize the work which simply has to be done to survive beyond drupal 8.4, so we don't end up with too many fronts open and too little time, otherwise I'm all for hacking our way out of this situation via radical moves — but in steps which will |
| [05:21:26] | <ergonlogic[m]> | I'm not sure exactly what that'll look like yet, but I've identified it as a possible sticking point |
| [05:23:32] | <ergonlogic[m]> | I believe in-place upgrades and incremental implementation will add significantly to the work-load, tbh |
| [05:24:41] | <ergonlogic[m]> | but, if we have a clean new implementation, we could perhaps build adapters to work with the existing front-end, and/or console-based backend |
| [05:25:28] | <ergonlogic[m]> | I'm concerned that we'll carry along too much baggage if we try to go incrementally |
| [05:26:46] | <ergonlogic[m]> | also, bear in mind that NG isn't starting from scratch. I already have a number of components in place from work last winter and spring |
| [05:28:00] | <helmo42[m]> | It's a sortof a choice between two evils ... incremental is more work for US Aegir maintainers, but a fresh start is more work for US as server administrators |
| [05:28:14] | <ergonlogic[m]> | these include a Vagrant-based dev env, a D8 profile, basic queue implementation, translatable, revisionable, fieldable custom entity, 100% test coverage, etc. |
| [05:28:22] | <memtkmcc> | hmm.. but what is the alternative? the drupal way? so, forget upgrade path and offer users the tool to set up new Aegir system and migrate/import sites? I have already seen too many 2.x servers which never upgraded to 3.x, despite working, albeit risky upgrade path |
| [05:29:02] | <ergonlogic[m]> | I think the import option is reasonable here |
| [05:30:06] | <ergonlogic[m]> | and that it's worthwhile to make that as easy as possible |
| [05:32:23] | <ergonlogic[m]> | an incremental upgrade plan is likely to take much longer, and have to handle more edge-cases |
| [05:32:38] | <memtkmcc> | developing well tested migrate tool could be a safe and clean way to avoid overhead on our side, plus, while upgrades in place sound good, in reality people tend to freak out and rather stay on the ancient version than risk to break live system, being constantly reminded the you should not upgrade your site in-place :-) |
| [05:32:55] | <ergonlogic[m]> | right |
| [05:33:10] | <helmo42[m]> | :) |
| [05:33:53] | <ergonlogic[m]> | we could, for example, add a "redirect" service (in aegir3), to simplify mass migrations |
| [05:34:14] | <ergonlogic[m]> | to reduce downtime and effort with DNS cut-vers |
| [05:34:20] | <ergonlogic[m]> | cut-overs |
| [05:34:49] | <ergonlogic[m]> | remote-importNG could look at platforms, and build equivlents in AegirNG |
| [05:34:49] | <helmo42[m]> | sure, I've done that with mod_proxy_http years ago |
| [05:35:43] | <ergonlogic[m]> | with those, we could have AegirNG slurp up whole platforms of sites, rather than handling each individually |
| [05:36:19] | <ergonlogic[m]> | but it'd all be isolated within a component that we could then easily deprecate eventually |
| [05:36:38] | <ergonlogic[m]> | without compromising the overall architecture |
| [05:38:50] | <memtkmcc> | in BOA we already have tools to migrate Aegir systems between machines, with clean Aegir install on the new machine following codebases and sites import, and we have used it already a few years back to migrate all systems between data centers in 6 facilities, so I know it can be done and can work very easily, if well tested, it even manages automatically created http/https proxies per site, so there is no do |
| [05:38:50] | <memtkmcc> | d-only mode enforced |
| [05:39:03] | <memtkmcc> | You can check it out at https://github.com/omega8cc/boa/blob/master/docs/MIGRATE.txt |
| [05:39:35] | <ergonlogic[m]> | also, in keeping with the Unix Philosophy, the basic components of AegirNG should be small re-usable "tasks", like "write an nginx vhost" or "provision a mysql database" |
| [05:39:55] | <ergonlogic[m]> | excellent! |
| [05:41:08] | <ergonlogic[m]> | writing those tasks should be the largest share of work to host a new type of application (e.g. wordpress) |
| [05:41:35] | <ergonlogic[m]> | but it should end up being pretty easy, if we build the AegirNG API correctly |
| [05:43:04] | <ergonlogic[m]> | basically, add a new task bundle, with fields for each variable that we'll need on the backend, then export that as a Feature. Then add a small Ansible role that consumes those variables and accomplishes the limited task |
| [05:43:32] | <ergonlogic[m]> | these are then combined into "operations", such as install, backup, update, etc |
| [05:44:12] | <ergonlogic[m]> | and a collection of such operations essentially defined a "platform" or a "site" |
| [05:45:37] | <ergonlogic[m]> | but writing a vhost should be largely similar for wordpress as for drupal, just with a couple differences in includes |
| [05:46:17] | <ergonlogic[m]> | in both cases, we need to specify a URL and aliases (for example), and reload the web server |
| [05:47:34] | <ergonlogic[m]> | but a common task could accept different templates, similar to now, just much more cleanly |
| [05:48:03] | <ergonlogic[m]> | because, you know, Ansible |
| [05:50:40] | <ergonlogic[m]> | anyway, I plan to push this as far as I can over the next week or two, since I'm between contracts |
| [05:51:09] | * memtkmcc has quit (Ping timeout: 246 seconds) |
| [05:52:23] | * memtkmcc has joined #aegir |
| [05:53:16] | <memtkmcc> | Well, I managed to convince myself that we shouldn't wrestle with incremental upgrade path, because *this* could cause more work and problems to solve than focusing on rewriting Aegir backend to be Drush-free. I still think we should be careful to keep the scope of work realistic in the timeframe we are dealing with. |
| [05:53:50] | <ergonlogic[m]> | agreed |
| [05:53:59] | <helmo42[m]> | ack |
| [05:54:09] | <ergonlogic[m]> | I think an MVP would be basic Drupal 8 platform/site support |
| [05:54:47] | <ergonlogic[m]> | including basic CRUD plus back/restore |
| [05:54:53] | <ergonlogic[m]> | backup/restore |
| [05:55:34] | <helmo42[m]> | I think we do need atleast one one Drupal thing next to D8 to prevent us from acidentally building in new Drupalisms |
| [05:56:04] | <ergonlogic[m]> | I think the new architecture avoids that already, but I see your point |
| [05:56:30] | <ergonlogic[m]> | maybe something simple (but still quite different) like a static site generator |
| [05:56:37] | <bgm> | you can count on me to try to implement WP/CiviCRM support asap :) |
| [05:57:03] | <ergonlogic[m]> | indeed, CiviCRM is critical to me too |
| [05:57:33] | <memtkmcc> | You can count on me to implement GravCMS support :-) |
| [05:58:14] | <bgm> | cool, I like this plan :) |
| [05:58:30] | <ergonlogic[m]> | ooh, GravCMS looks shiny! |
| [05:59:15] | <memtkmcc> | indeed! |
| [06:00:40] | <ergonlogic[m]> | so, fyi, I've been stuck on getting the queue to work in CI tests, despite working very nicely locally |
| [06:01:20] | <ergonlogic[m]> | so I'm going to disabled CI tests that require the queue, which'll include most of the newer task-oriented stuff |
| [06:02:00] | <ergonlogic[m]> | and focus instead on keeping high levels of code coverage running tests locally |
| [06:02:47] | <ergonlogic[m]> | eventually, we can figure out the queue issues on gitlab-ci, and re-enable the full suite |
| [06:03:03] | <ergonlogic[m]> | but I don't want to waste any more time on it right now |
| [06:06:25] | <memtkmcc> | sounds reasonable, we don't have time to waste on things which can wait a bit |
| [06:09:59] | <helmo42[m]> | I've gotta go soon... lets continue the 'short term' discussion when jonpugh_ is also around. |
| [06:10:33] | <ergonlogic[m]> | I'll post a link to the recent convo to the mailing list, so that everyone's up-to-date |
| [06:10:45] | <ergonlogic[m]> | hefring: log pointer |
| [06:10:45] | <hefring> | http://hefring.mig5.net/bot/log/aegir/2017-10-05#T659807 |
| [06:11:12] | <memtkmcc> | good idea, thanks! |
| [06:11:19] | <helmo42[m]> | I'll be happy to setup my own test NG server once we get there ... but I see my main role in maintaining 3.x/4.x as long as we need to and pave the migration path |
| [06:13:51] | <ergonlogic[m]> | sure. I'm planning to press forward as far and fast as possible down this path |
| [06:14:07] | <ergonlogic[m]> | once we have a rough prototype, it'll be easier to see where to go from there |
| [06:17:51] | <memtkmcc> | we seem to have both a consensus and a good momentum on this, plus some code we can build the prototype upon, at least up to the clarification point, where it will become easier to make choices and push forward |
| [06:20:13] | <ergonlogic[m]> | agreed |
| [06:23:24] | * memtkmcc1 has joined #aegir |
| [06:25:38] | * memtkmcc has quit (Ping timeout: 255 seconds) |
| [06:39:59] | * shaneonabike1 has quit (Ping timeout: 258 seconds) |
| [06:41:30] | * shaneonabike1 has joined #aegir |
| [06:44:38] | * memtkmcc1 has quit (Quit: Leaving.) |
| [08:46:21] | * memtkmcc has joined #aegir |
| [09:44:08] | * memtkmcc has quit (Quit: Leaving.) |
