Smokes your problems, coughs fresh air.

Author: Rowan Rodrik (Page 8 of 27)

Rowan is mainly a writer. This blog here is a dumping ground for miscellaneous stuff that he just needs to get out of his head. He is way more passionate about the subjects he writes about on Sapiens Habitat: the connections between humans, each other, and to nature, including their human nature.

If you are dreaming of a holiday in the forests of Drenthe (the Netherlands), look no further than “De Schuilplaats”: a beautiful vacation home, around which Rowan maintains a magnificent ecological garden and a private heather field, brimming with biological diversity.

FlashMQ is a business that offers managed MQTT hosting and other services that Rowan co-founded with Jeroen and Wiebe.

NFSN PHP file write permissions in safe_mode

I’ve been causing some (security) concerns for myself by thoughtlessly using the dreaded 777 permissions for upload directories to allow the various PHP-based websites that I host at nearlyfreespeech.net to write files there. What this drastic anti-security measure didn’t allow me to is to manage these uploaded files through SSH (and SCP/Rsync). In the chroot jail which I’m allowed to enter through SSH, I am ‘me’, while the files created from PHP end up being owned by user ‘web’. However, for some reason these files didn’t get owned by group ‘web’ of which the ‘me’ user is a member. Also, I got into trouble with new directories that were being created by the upload scripts.

Writing files in PHP more securely

When I make a very basic test script in PHP, which writes a new file to a 777 directory owned by ‘me’, the files simply end up being owned by web:web with 775 permissions. To make this work a little more securely, I have to change the authorization for two entities:

  1. The directory to which I want to write needs to be group writable (775) and owned by group ‘web’.
  2. The PHP file that does the writing needs to also be owned by group ‘web’.

This will also allow the PHP file to write files in subdirectories that it creates. In the 777 scenario above, it would be possible to create these dirs, but not to create files within them. The PHP safe_mode restrictions in effect won’t allow a script owned by user ‘me’ and another group than ‘web’ to write files in a directory owned by ‘web:web’ and 775 permissions set.

All in accordance with the advice on writing files in PHP by the NFSN team.

Application headaches

There’s a little more to it, though. The stuff that I uploaded through MediaWiki and WordPress with my super-liberal 777 permission set on the upload dir (owned by ‘me’) somehow never ended up with the same group write permissions as the files in PHP test described above.

WordPress on NearlyFreeSpeech.Net

Uploads created by WordPress did end up with the permissions of the beast (666) set. However, directories created by WordPress (the year/month no. subdirectories) ended up with ‘web:web’ ownership, which regardless of their 777 mode, didn’t allow PHP in safe_mode to create any files within these directories. This is easy enough to solve by changing the ownership of the PHP files doing the writing to group ‘web’. Of course, this is best coupled with making the same changes to the upload directories and also changing the mode of these to 775.

The WordPress installation notes on the NFSN member wiki [for members only] has some more details.

I reviewed the code in WordPress responsible for writing files and I noticed that, whether creating files or directory, it actually looks at the permissions of the parent directory to decide on the mode of the newly created entity (using something along the lines of “$new_file_perms = $parent_perms & 0000666” for files and “$new_dir_perms = $parent_perms & 0007777” for directories).

MediaWiki on NearlyFreeSpeech.Net

The NFSN member wiki offers some NFSN-specific instruction for setting up MediaWiki [in their walled garden].

As with WordPress, I changed the ownership of the top-level PHP files and the upload directories to group ‘web’, as well as changing the permissions of the upload directories to 775.

However, uploaded files are being created with the mode 644 instead of 664. This is hugely annoying, because, still, I’m not allowed to access these files through SSH. I have yet to find out how I can best remedy this. Probably, I’ll end up with writing a simple PHP script that I can call just to chmod everything within the upload directory when the urge to manipulate these files strikes me.

Another beef with MediaWiki is that it creates subdirectories in the uploads directory with mode 777 instead of looking at the mode of the parent dir as WordPress does so neatly.

WordPress admin bar and absolute CSS positioning

WordPress 3.2 introduced an admin bar, which is fixed at the top of the window when you’re logged in. An annoying side-effect that has been bugging me for some time is that, although this pushed down most of my content, HTML elements that used absolute positioning stayed in there old place. This didn’t look particularly good, but I left it hanging for a long time because it only affected logged in users (that is: either me or Halfgaar).

The fix is actually rather simple. I added the following rule:

body {:; }

It may seem non-obvious, but absolutely positioned elements are actually positioned relative to their relatively positioned ancestors. Well, just read the proper explanation at CSS-Tricks. I can’t explain this shit.

WordPress admin bar with correct positioning

How it was supposed to look

WordPress admin bar induced CSS positioning screw-up

How it really looked

Hobo Lobo of Hamelin

Hobo Lobo of Hamelin is worth looking at even if just for the novelty of how the story and artwork are digitally presented (something to do with a ‘parallaxer’). But, I’d much rather let the creator do the talking:

‘Hobo Lobo of Hamelin is a thing by a dude, who’s all like, “I’M GONNA MAKE A THING.” And then he did. Or is doing. Or, you know, whatever. This dude can be found on the internet. He kerns alphabet soup and rearranges window dressing in the aether to put food on his family.’

Nakedness

2018-01-03. I’m making this backpost because I just unpublished the following blurb from www.bigsmoke.us, where it accompanied the link to this blog’s Nakedness category that has since been superceded by the Tribulation.

I am a naked ape. You are too, but some of you are in denial. Privacy is dead. It was assaulted by modern communication technology and burried by Facebook (and you dug your own hole). How are you going to hide your true monkey nature when you’re fully transparent?

Everybody is still frantically protecting their image, but it’s fucking useless to post only your heroics, since everybody can see that everyone is the same. No-one can ascend their nature. No-one can ascend nature, period. We are nature and we are natural, no matter what type of holy shit we sprinkle ourselves with.

Sometimes, I feel inclined to share with you a rawer version of myself.

My zeroth year at university

Maybe my biggest accomplishment to date—maybe my only real accomplishment, if your glasses are so colored by society’s standards—has been to be accepted to the University of Groningen as a fulltime biology student. To apply, I had to send my curriculum and a letter of motivation. Which motivation? I wasn’t so sure that I’d like to be a student. Actually, I had been quite certain for most of my adult life that I really did not want to study and waste all that precious time for a few crums of knowledge.

But, I overdosed on spacecake and was having a bad trip. I was already depressed. My life hadn’t worked out. I hadn’t turned out to be the type of person that I wanted to be. None of the success. None of the happiness. Little satisfaction. Just some stubborn fantasies about how cool me and my life would be if only…

The physical and mental stress caused by the fear that underlies most major depressions overtook me, so terribly afraid of what others—that’s you—might think of me. This sensation wasn’t new. What was new was a lasting awareness about the extend to which this social anxiety directed my life and a stronger sense about how this might have affected my major life decisions. I felt (more than that I thought) that, maybe, I could try the normal life of a college student.

At the same time, I was very doubtful, because I had occassionally tried to fit into the constraints of society. It never fitted. I had to always give up on the straight path. So why would this work?

I did know that I was interested in biology and—by myself—I never really dug into it, apart from enjoying a Attenborough documentary or two. So, I investigated my options and decided to apply for university.

The next couple of months are a blur of learning, intensifying bouts of depression, despair and the occasional glimmer of hope. Never having finished even one of the lowest level of high-school, I had to face a colloquium doctum, where my knowledge of mathematics, physics, chemistry and biology would be tested.

During the first examination, only my understanding of biology was sufficient. My math, physics and chemistry were terrible (a 2.5, a 2, and a 1 (out of 10) respectively), just above elementary school level.

I had only two attempts left to be in time to start studying after the 2011 summer break. The year after, I’d be 30 and no longer eligible to state support as a student.

During the next attempt, I fluked all remaining three subjects (although physics had turned into a 4). Then, the last attempt approached. I was nervous as hell, and felt ill-prepared at best. I was high on sleep-deprivation during the physics part. Yet, I was confident. Mathematics went terrible. It was mostly calculus and the statistics part was also much harder than the practice exams that I’d used.

So I resigned in my head, because I was certain that I had failed math. I decided I wanted to know how much chemistry had improved since my last attempt, though. (It was so bad then that it wasn’t even graded.) Surprisingly, chemistry went somewhat okay. At least I had made a somewhat informed attempt at an answer on most questions.

Came my grades for math and physics: a 5.5 and a 5.9. How was this possible? I was already planning to go back to France to work with my brother. A 5.5 was exactly sufficient to meet the requirements.

The first semester would start in a week. But I’d have to wait a week for the chemistry grade. This was thrilling, in a good way and a bad way. Finally, the grade came in, just in time for me to know if It’d make sense for me to come to university the next day for all the introductions that would take place.

The next day I was sitting in a lecture hall, filled to the brim with hundreds of 18-year-olds. In just a couple of months I had gone from a 0 (that’s a zero) on chemistry to a whopping 7.8!

Trying to reduce MySQL InnoDB disk usage after major reduction of data

So, two days ago, I tried to shrink my MediaWiki database and it almost worked, except the MySQL process wouldn’t shrink along with it.

Of course I tried all the obvious things such as dropping the database, stopping and restarting the process, followed by reloading the database. Optimizing tables, altering tables, all the obvious. But, to no avail, because there’s this bug. Well, technically it’s not a bug. Like most of MySQL, it has been “designed” to be counter-intuitive, non-standard, riddled with special cases, exceptions, work-arounds and duct tape. So, the “bug” is really just a feature request from dumb people much like myself who want stupid things like a database that becomes smaller in size when you delete most of your data.

I should really move to a virtual private server environment, where I can just run a real database (PostgreSQL), but I’m still on NFSN, whom (besides the sky-rocketing storage costs as of late) have given me no reason to complain so far.

I thought I’d ask them to help me out.

Recently, due to inattention to spam, one of my wiki databases has grown to well over 10 GiB. After finally getting around to removing most of this spam and tweaking some settings to reduce the table size from over 11Gig to a couple of MiB, I thought my Last Reported Size would go down accordingly.

But no such luck. Apparently it’s a MySQL issue, with the only solution being to drop the offending database (after a dump, of course), stop the MySQL process, remove the offending table, restart the process and then reload the database.

Instead, you could use the innodb_file_per_table option, which is enabled by default on MariaDB.

Without that option set, OPTIMIZE, ALTER TABLE and all that stuff will do nothing to reduce a table size. It’s one of those issues which the MySQL devs are really not interested in solving: http://bugs.mysql.com/bug.php?id=1341

I hope you can help me out with this, either by setting the innodb_file_per_table option or by removing all my database files. In the latter case, I’d hope you ask me to green light this first so that I can make some other data size reductions in various databases before I make a final backup.

But then I thought better of it, when I learned that—contrary to my expectations—the option really was enabled:

SHOW VARIABLES LIKE ‘innodb_file_per_table’;

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| innodb_file_per_table | ON    | 
+-----------------------+-------+
1 row in set (0.00 sec)

So the option was enabled. I had to look elsewhere for an answer, which made me decide to do the following.

OPT="--host=myhost --user=myuser --password=mypassword --skip-column-names --batch hardwood";
mysql $OPT --execute="show tables" | grep -v mw_searchindex | tablename;
  mysql $OPT --execute="alter table $tablename ENGINE=InnoDB; optimize table $tablename"

Needless to say, this didn’t help. So I wrote the support request for my hosting provider:

Recently, due to inattention to spam, one of my wiki databases has grown to well over 11 GiB. After finally getting around to removing most of this spam and tweaking some settings to reduce the table size of the most offending table from over 11Gig to a couple of MiB, I thought my Last Reported Size would go down accordingly.

Since you have the innodb_file_per_table option enabled (the default in MariaDB and confirmed by “SHOW VARIABLES LIKE 'innodb_file_per_table';”), I’d expect “ALTER TABLE mw_text ENGINE=InnoDB” to shrink the table (the command should move the table to its own file, even if it wasn’t already in its own file before the move to MariaDB). It didn’t. Last Reported Size is still approximately the same. Dropping and reloading the entire database didn’t do much to help either.

I suspect that the problem is that that the old shared file still exists and that this file won’t be deleted even if there are no more tables in it with the only solution then being to dump the database, drop it, remove all its files and then reload the dump.

Anyway, I’d like it to be so that my database will actually shrink when I delete stuff and the way I understand it this should actually be possible thanks to innodb_file_per_table. If it’s the same old story as without innodb_file_per_table, that would just be awful, because then I’d need your intervention every time that I’m trying to reduce my database process size.

I hope that you can somehow reload the database and remove the bloated
ibdata1 file.

Now, I’m just waiting patiently for their response while my funds whither…

« Older posts Newer posts »

© 2024 BigSmoke

Theme by Anders NorenUp ↑