snf.wsring.net

Friendica update script (for git-based installations)

I've been using this script for ~3 months now. I added a few additional steps based on input from @Steffen K9 🐰 not too long ago. I'm sure it's still not perfect, but it seems to be working just fine for me, so I thought I might as well share it on here and possibly help others!

I find this much simpler than manually extracting archives, fixing permissions, overwritting configs and relinking my personal additions to the site with every update. I also include some tips in the file for dealing with branch changes and such (again, mainly thanks to what I learned from @Steffen K9 🐰 ). I know it's overly verbose, but I'm sure that's no surprise to any of you that have chatted with me on here :P

Instructions are in the comments, although it's probably obvious from the script, it's fairly simple. But I want it to be clear to folks who might not be all that familiar with bash/git/system administration. Anyways, here it is:
#[url=https://snf.wsring.net/search?tag=!/bin/bash]!/bin/bash[/url]
#
# Friendica Update Script
# by Adam Gaskins <self@adamgaskins.com>
# updated 06.17.19
#
# Description: Pull the latest friendica and friendica-addons git changes, then 
# do post-install and maintenance stuff. It is not a great idea to run this 
# script from cron or a systemd timer if you are using the 'develop' or RC 
# branches.
#
# Usage: Save this script somewhere like /usr/local/sbin/friendica_update.sh 
# (but it shouldn't really matter). Then make it executable with something like 
# 'chmod ug+x friendica_update.sh'. Adjust the variables in the 'USER CONFIG' 
# section to match your setup and preferences, and then execute this script by 
# running './usr/local/sbin/friendica_update.sh', or whatever method you prefer. 
# This script is simple and straight forward, but I still added comments for 
# variable explanation, and wherever there wasn't an 'echo' line to explain what 
# was happening, so it should be extremely clear what is going on here!
#
# Tips: These are just a few tips that I found useful, being a relative n00b 
# with git myself. It's just a couple of basic things that one should know how 
# to do, especially if you're going to use a branch other than 'master, like a 
# release candidate (RC) or the 'develop' branch.
#
# To change branches between master, develop, etc. run a command similar 
# to the one below, replacing 'master' with the branch name you wish use. Also, 
# replace 'www-data', if needed, with the user/group that your webserver expects 
# the files to be owned by. The sudo part is very important for git commands 
# that might change or create files, to avoid making the files inaccessible to 
# your webserver!
#   sudo -u www-data -g www-data git checkout master
#   sudo -u www-data -g www-data git checkout develop
#
# To view all available branches on the remote repo run:
#   git branch --list --all --verbose
#
# Sometimes branches are removed on the remote repo, such as those for release 
# candidates that have been merged to master. You may see a message like this: 
#   > Your configuration specifies to merge with the ref 'origin/2019.06-RC' 
#   > from the remote, but no such ref was fetched.
# This means you need to change to another branch, probably 'master' in this 
# case. You'll want to use 'branch' and 'checkout' command shown previously to 
# find and checkout a new branch. You can then do a 'pull' to get your local 
# repo in sync with the remote branch, but add '--prune' this time to remove 
# any old branches that are no longer used!
#   sudo -u www-data -g www-data git fetch --prune
#
# The above commands do NOT need to be run every time, so just do this stuff 
# manually, as needed! like when you wish to switch branches, for example. If 
# you're running an open public node then you probably just want to follow 
# 'master' and never really change branches anyways, but sometimes the RC 
# branches are useful! It all just depends on your confort zone, and how 
# diligently you are keeping an eye on your node.
#
# Oh, and DO NOT FORGET to run these commands in both your main Friendica repo 
# AND the addon/ directory! These are two independent repos and they should be 
# on like branches!
#
# TODO: add logging, error handling, verbosity control
#
# Thanks to Steffen K9 for sharing some tips that helped me write this script!
# Ref: https://social.linuxlusers.com/display/0b6b25a8-705c-e978-6135-0fd801926751
#
# Contact me if you have any corrections, concerns, or suggestions! I'm 
# certainly no wiz at bash nor git, so I'm happy to learn of ways to improve 
# this!

#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]
#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url] USER CONFIG #[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]

# A message shown to users during the update proceedure
MESSAGE="This system is currently undergoing a routine update. Our sincerest apologies for any trouble this may have caused you, but it should only take a moment to complete, then the system will resume normal operation!"

# Path where Friendica git repository resides
BASEPATH='/var/www/friendica/git-repo/'

# Run processes as this user (and group)
RUNAS='www-data'

# Path to php executable, or just 'php' to use the system default 
PHPEXEC='php'

#[url=https://snf.wsring.net/search?tag=#]#[/url] END OF USER CONFIG #[url=https://snf.wsring.net/search?tag=#]#[/url]
#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url]]#[url=https://snf.wsring.net/search?tag=#]#[/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url]#[url=https://snf.wsring.net/search?tag=#]#[/url][/url][/url]

echo -e "Starting update routine for Friendica ($BASEPATH) on $(date)"
pushd $BASEPATH >/dev/null

echo -e "\nEntering maintenance mode:"
sudo -u $RUNAS -g $RUNAS $PHPEXEC bin/console.php maintenance 1 "$MESSAGE"

echo -e "\nUpdating $(pwd):"
sudo -u $RUNAS -g $RUNAS git pull

# addon repo
pushd addon/ >/dev/null

echo -e "\nUpdating $(pwd):"
sudo -u $RUNAS -g $RUNAS git pull

# return to main repo
popd >/dev/null

echo -e "\nInstall/update dependencies in $(pwd):"
sudo -u $RUNAS -g $RUNAS $PHPEXEC bin/composer.phar install --no-dev

echo -e "\nUpdate the database (if needed):"
sudo -u $RUNAS -g $RUNAS $PHPEXEC bin/console.php dbstructure update

echo -e "\nRun post-update tasks:"
sudo -u $RUNAS -g $RUNAS $PHPEXEC bin/console.php postupdate

# Status report
echo -e "\nGit status (main):"
sudo -u $RUNAS -g $RUNAS git status -bsu
echo -e "\nGit status (addon):"
sudo -u $RUNAS -g $RUNAS git -C addon/ status -bsu

# Return to normal mode
echo -e "\nExiting maintenance mode:"
sudo -u $RUNAS -g $RUNAS $PHPEXEC bin/console.php maintenance 0

# Final pop to where we started
popd >/dev/null

echo -e "\nThis script has completed. Please check the output (above) for potentially important messages or issues!"

exit 0

P.S. The text is much easier to read once copied and pasted in a file, all comments are wrapped at 80 characters. I'm sorry the comments are kind of a mess here - at least on my screen using Frio. I would have thought 80 characts would fit :\

!Friendica Support
!Friendica Admins
!The Linux Luser Group
utzer Friendica
@Adam Gaskins could we just add that to the repository as well?

One addition could be to write all output to some log, preferably in a block, then this could be the friendica log and would still be possible to find it easily.

My script was much easier but covered mostly the same steps, @Hypolite Petovan helped me with that a year ago.

I like the script here and would love to see that in the repository as I said.

#!/bin/bash
echo=$(command -v echo)
git=$(command -v git)
SCRIPTPATH="$( cd "$(dirname "$0")" ; pwd -P )"
http_path="$SCRIPTPATH/../"
friendica_log="/path/to/friendica/logfile.log"

cd $http_path
"$git"  pull
#bin/composer.phar update --lock
bin/composer.phar install --no-dev
cd addon
"$git"  pull
"$echo" "Git pull and bin/composer.phar install successful at $(date)" >> $friendica_log
exit 0
@utzer @Hypolite Petovan Yes! This is actually on my TODO items in the script, haha. I was just lazy. Do you think it should use the actual file that friendica uses, the log.out? Or it's own update.log, for example? I guess use could specify either, but I'm just imaging which would be best to suggest in the script. I'll try to update this today! Thanks for the suggestion!
@utzer I like your much more consice script. How do you take care of file ownership? Do you just run the entire script via 'sudo -u www-data /path/to/script.sh (or whatever the case may be)?
I do like the idea of have no user configuration. So you just place this in a subfolder of your friendica root?
utzer Friendica
@Adam Gaskins yes, I place it in a hidden subfolder and run it with sudo -u as you wrote.
utzer Friendica
@Adam Gaskins this way you do not need to set a path and you don't need to set a user/group.

I am not sure what is the better way.
@utzer I just know that in my setup nginx will be unhappy if the files are not owned (or at least in the group) 'www-data'. if I just run 'git pull', as any other user, then any new files will end up with the ownership of the user (and their group membership) which ran the command. @Steffen K9 🐰 handled it with sudo too, similar to the way I do it in the script, but he also posted an example using a final chown to fix things, so it seems to be a fairly common issue to avoided. I'm always learning new stuff, so I don't claim I'm right here, haha. Maybe it's just not an issue due to the way you run the script? Or how you have your webserver configured?
@utzer doh! ignore my comment if the delete doesn't work properly! I didn't see both comments, but rather just the second, and was confused. This makes perfect sense now. Sorry about that, haha.
utzer Friendica
@hoergen it is in both scripts, my minimalistic and @self@social.linuxlusers.com's as well?!
Savi Friendica
Hallo
Als Mitglied des größten mystischen Ordens der Welt (Illuminati) lade ich Sie ein, sich unserem mystischen Orden anzuschließen, der sich in den letzten Jahren öffentlich der Welt zu offenbaren begann.Weitere Informationen zur Mitgliedschaft erhalten Sie unter dieser Adresse mail:
illuminatisaccueil208@gmail.com
@hoergen @Steffen K9 🐰 Yeah, I do like this idea. I'm not sure if you noticed, but probably the bulk of my pre-script comments deal with explaining how to do this manually. I just use git branch -l -a -v to show everything in one command.
But I agree, having it in the script would be nice too. I was just torn though, because I figure they had to already get it installed via git to use this script, but even from my own experience, it was sometime after I was checking out repos before I learned to change branches (I've been very lazy in getting up to speed with git, ha). So yeah, I guess this would be a nice feature! Thanks @hoergen !
@Savi @hoergen @utzer OMG I've waited my whole life for this invite in to your secret order! I'll be contacting you right away for my membership card!
This is what I actually do for regular updates:

##Update Friendica Community Forum

echo "Forum Update"

cd /var/www/html/frforum/
sudo -u www-data HOME=/var/tmp/www-data git pull

cd /var/www/html/frforum/addon/
sudo -u www-data HOME=/var/tmp/www-data git pull

cd /var/www/html/frforum/
sudo -u www-data COMPOSER_HOME=/var/tmp/composer bin/composer.phar install --no-dev

##Restart Apache

sleep 2s

service apache2 restart

service php7.2-fpm restart

echo
echo "Forum has been updated"
echo


It's very simple but it works.
@Steffen K9 🐰 I have a couple of questions:
1) How important is the HOME/COMPOSER_HOME variable in the sudo command? I didn't use it my script just because neither of those paths exist on my server, and I wasn't sure they were correct for all setups. Maybe they are just needed during runtime of this command?

2) Why do you restart apache and php? I've never done this during my friendica updates. Is the apache restart just to reload .htaccess? Since I use nginx I handle that sort of thing in my site configs. Is it really needed to even update the htaccess file? I'm not sure about php either - I use fpm as well and always just leave it running after these Friendica updates.

So far, so good, without these bits, but I don't want to ommit anything important that might cause an issue!

I'm not in any way claiming these things aren't needed, but rather I just don't understand what they are doing in these circumstances (and I have made some efforts to figure it out). I'd just like to be able to explain the steps in the script clearly, since I'm sharing it, and of course grok everything fully myself! haha

Thanks so much @Steffen K9 🐰 ! You've been a huge help on this already!
Well 'www-data' has no real home directory. git and composer are both trying to place a directory and cached files in the homedir of the user running them.
At least that's true for me and my system. So I created temp directories for both and use them in environment variables when running the commands.

The restart of Apache isn't necessary. But if you use PHP-FPM I'd advise to at least restart that. PHP is using some code caches and other caches. A restart is cleaning that up. I do that to be sure everything is fresh. :-)
Great info! I think it all makes sense now and I can revise my script without any 'grey' areas! Thanks so much for clearing that up @Steffen K9 🐰 !
@Steffen K9 🐰 Just a quick question to see if I'm going in the right direction with this line of thought - If I wanted to figure out if an update might include a change to the database *before* proceeding with the fetch & merge (pull), would a diff on the existing local and latest remote database.sql file be the way to go about doing this?