Category — PHP

Ran into a problem today compiling PHP with IMAP support. Was getting this error in the config.log:

undefined reference to `auth_gssapi_valid’

I then found this thread: http://bugs.php.net/bug.php?id=21286

Which had the solution to the problem. Simply add:

–with-openssl

To your php configure and it should now work.

UPDATED: Ok this did not end up working. It worked for a good 6 hours or so, then things with the database server got worse than before. Will keep you updated on a fix.

Had been trying to troubleshoot this error for what seemed like eternity. It happened every so often, not at all following any type of pattern. I did all the standard googling of this term and found nothing but useless information. FINALLY I found one website after much length of surfing that has fixed my problem. The solution is to increase your max_allowed_packets in your my.cnf config file for mysql. After making this change I haven’t seen this error once.

Here is the site that had the solution. Lost connection Solution

I recently implemented a really easy solution for tracking Fatal errors in PHP on my live sites. It is really helpful if you have a large codebase, or manage a large number of PHP sites. What I am doing first is to report all php errors to a log file, then watch the log file for Fatal errors then email them to myself.

So first step is to turn the error reporting to a log file in PHP (are you displaying yours to screen? cause on production servers you damn well shouldn’t!). To do this edit your php.ini file and find the line with:

error_log = /something;

now we will change this line to where we want out error log to go. Lets say we want our php errors put into a file called php_errors located in the /var/log folder. So our php.ini line would look like:

error_log = /var/log/php_errors

OK while you are in the php.ini file you might as well stop showing errors to the screen, to do that find the line:

display_errors = On

and change it to:

display_errors = Off

OK now we need to restart apache for that to take effect. After apache is restarted you should now start to see errors in the file /var/log/php_errors, and you should no longer be seeing PHP errors on your screen.

Now the next step of this is to report Fatal errors. For this I use SWATCH . I will let you figure out how to install it (pretty simple, make, make test, make install).

After SWATCH is installed lets make a config file for it to report on PHP Fatal errors. So I create a file called:

/etc/swatch/php_fatal_errors.conf

Swatch uses perl regular expressions to report on so to catch PHP fatal errors my config looks something like this:

# watch out for PHP Fatal Errors
watchfor /PHP Fatal error/
mail addresses=myemail\@address.com,subject=Fatal Error On Website

Notice that I escaped the @ symbol, this is because in perl the @ symbol is used for arrays. Anyway that is it. You can now start swatch by running:

swatch –config-file=/etc/swatch/php_fatal_errors.conf –tail-file=/var/log/php_errors &

This will start swatch in the background using the config file you have just created and will montior the PHP error log file we setup above. Now everytime there is a PHP fatal error on your site you will get an email. You may want to play around with the swatch config, specifically with the throttle directive as it will allow you to prevent yourself from getting overwhealmed with mass emails if you have a lot of errors.

Anyway that is it, a quick and easy way to monitor your live sites for PHP errors. While your code should be error free before you go live with it, you can’t always find all the errors, so this is an easy solution to tracking down those problems.

The other day I was analysing a bunch of different encoder/optimizers for PHP to determine what the best solution would be for a specific project. What they needed was the ability to encode our php scripts as well as cache/optimize our server to increase our site speed. They were currently using mmcache as our encoder/optimizer, however due it the lack of support and several bugs (especially for high trafficed sites) they needed to move to a different solution. I eveluated APC , Zend Platform w/ the Zend Encoder, Ioncube w/ php accelerator and MMCache. I used apache flood to do all of my tests. A few things to note. Zend was run with encoded files using the zend encoder. Ioncube/php accelerated used ioncube encoded files. MMCache was tested with mmcache encoded files. APC was tested with no encoded files as APC has no encoder. However I did test APC with Source Guardian and found after some preliminary test that they worked alright together. Also all encoders were setup using Apache 1.3.33 and PHP 4.4.0 except for mmcache which used Apache 1.3.33 and PHP 4.3.9. The reason for using the lower version of PHP for mmcache was because of the foreach bug with it and any later version of PHP.

I did 4 different tests ranging from 500 requests to 10,000 requests on over 20 different php scripts that all did different types of things (database access, filesystem access, straight output etc). Here are my results calculated in number of requests per second.

510 Requests
APC: 19.17
mmcache: 16.11
Zend: 19.47
Ioncube: 10.7

3000 Requests
APC: 19.63
mmcache: 19.69
Zend: 18.72
Ioncube: 10.82

5,000 Requests
APC: 19.6
mmcache: 19.51
Zend: 20.37
Ioncube: 11.06
None: 7.39

10,000 Requests
APC: 19.09
mmcache: 19.19
Zend: 18.93
Ioncube: 10.74
None: 7.14

So from these results you can see that APC and Zend show very similar results, yet cannot surpass the speed of mmcache. It must be a testiment to MMCache in that it hasn’t seen any new development since 2003 yet still is top of the field (shame about the bugs however). Lastly I was pretty disapointed with the results of Ioncube, only being able to produce about half the number of requests per second as Zend and APC.

As for which product they will choose I am unsure as of now. APC has the strong benefit of being free, however not having an encoder may make them lean in the direction of Zend.

Lastly to note the reason I never included eaccelerator in these tests is because it contains many of the same bugs that MMCache has regarding high load situations, mainly apache deadlocks. It is sad really because the people who need a good encoder/optimizer are those who are under high load situations. Anyway it doesn’t look like the eaccelerator crew really has it’s stuff together either way with a pretty inactive development team.

So take what you want from these test results, they are by no means is it the last word, but simply my findings after a few days of testing.

Yesterday I attend the PHP & Open Source Security Conference put on by OSEVENTS (formerly PHP West) at the HR MacMillan Space Centre here in Vancouver. I was very much looking forward to this conference as lately I have been working on making security a lot bigger issue in the code I am writing.

The first talk of the day was by Bruce Perens about security in open source development in general. We entered the conference room and Bruce was already up on the stage talking. He hadn’t started his presentation yet, but was spending the time talking about an issue that seemed very important to him, that was the issue of software patenting. Perens painted a very grim future of how archaic patent laws if (or when) applied to software will end up supporting big corporations and literally kill small/medium business as well as open source development. I will write more on the patent issues at a later time.

After everyone was registered and settled into the conference room Perens begin his first talk regarding security in open source. He talked about some of the good things the open source community had going for it such as “many eyes”, a commitment to fixing problems not hiding them, and the ability for organizations to make changes themselves that are needed.

One of the biggest issues Bruce brought up during his talk was regarding communication of information. The problem being an often lack of skills from the people within the community to communicate information to non technical people. This presents a problem in several aspects because it often gets the non technical user to go to other sources for information which often have wrong information, or will cause media to report on things they don’t fully understand and often over sensationalize. One way to combat this was for people within the community who can write and communicate well should use their ability and share their knowledge. He thinks that this needs to be done more than just on Blogs and advocates writing editorials or letters to the editor.

In Perens eyes another major problem in open source development that needs attention is that of tracking the identity of the people writing the code on open source projects. Perens thinks that in the future it is very possible for larger corporations to plant senior developers on open source projects in the attempt to sabotage the project and make it look bad in the medias eyes. In our current development paradigm this is a real threat Perens explained because the identity of the developers writing the code is unknown. A solution to this that Perens helped implement at Debian was that all developers had to digitally sign their code whenever they checked it in. Whenever there was a meet-up of Debian developers they would then share their drivers license and passport information with the other developers to prove their identity. In this way they were able to link identities to all of their developers and link them to their signatures. In this way if their was ever a major exploit or back door programed into the code they would have a direct link to the person who programmed this and would be able to prosecute them swiftly.

While Perens talk was not PHP specific, I still quite enjoyed it. He brought up several issues I hadn’t really paid attention to in the past and other issues I wasn’t even aware of. Perens also impressed me with his vast amount of knowledge of Open Source in general.

Next to talk was Christian Wenz from Germany. His talk was titled “New Trends in Web Hacking”. Christian’s began by first going over some older security problems that still face many php applications such as Cross Site Scripting (XSS) and SQL injections and the importance to protect against these attacks. He then went on to talk about new problems that are arising such blog spamming. Christian used good use of code examples to quickly show how these hacking attempts were being performed. Christian obviously knew a lot about the subject matter and gave a good talk. I was a little disappointed however that his talk was only 45 minutes. It seemed a bit rushed to me and it didn’t really give him time to talk indepth on solutions to the problems, but rather had him saying things like “This is bad.” and moving on. One thing that I really enjoyed about Christian was his sense of humor, it reminded me a little of Cal Henderson’s humor during his talk at the web services conference. One of Christian’s funnest jokes of the day was when he showed us how people are baiting users to pass Text CAPCHA’s for them by promising ‘hardcore XXX porn’. “Because I hear when people are up late at night and are bored they like to surf for porn. So I have heard.”, too funny.

Lunch was served after Christians talk. It was a basic sandwich lunch, which I suppose sufficed. Actually the bread they had was very excellent. I was a however a little disappointed with the lack of anything else. They could have at least provided cookies, juice and other such snacks to round off the meal. Regardless the sandwiches were good.

After lunch the next speaker was Chris Shiflett, someone who I really looked forward to hearing. His talk was title “PHP Security Audit HOWTO” and was a practical guide on how to perform in house security audits on other team members code. Chris started by talking about the two most important things a PHP developer should do, and that is to Filter Input and Escape Output. This is something Chris talks very often about on his blog and he reiterated the importance of it here. Below is a summary of key points I thought were interesting:

• When filtering input never modify invalid data to make it valid. A whitelist approach should be used (Reject everything except that what is specified).
• You should never trust what is in $_SERVER
• Escaped output should never have to be undone. If it does it means you have done something wrong.
• Some key things to turn off in your php.ini file: register_globals, allow_url_fopen, display_errors, magic_quotes.
• When conducting an audit follow the process of finding where input happens (such as form data), then determine where output occurs (database interaction, output to html). Use the input as your starting point of your audit then trace your way back to where those variables are defined. When you find where the variables are defined you can then determine if they have been properly filtered/escaped before being used or outputted. Other things to watch out for is error suppressing, dynamic includes (make sure the variable is clean).
• Cookies should not be trusted.
• Only include files in your document root that you want accessible by a url.

Chris’s talk was very helpful and practical. It reiterated a lot of the security practices I have read about in the past and made me feel pretty good about the code I have been writing lately. I found Chris to be a very good public speaker, his talk was organized and complete and he explained things clearly and thoroughly. Mostly though I was blown away by his vast knowledge of security in general, he has a very strong grasp on security from a theoretical and a practical standpoint. I think Chris’s talk alone made this conference worth while for me.

Following Chris Shiflett at a conference dedicated to PHP security is a very difficult thing to do. The next person to speak was Tom Robinson a freelance developer from Vancouver. Tom took the stage and didn’t do much of introducing himself and sort of mumbled that the other speakers had already talked about a lot of what he was going to say. He then started to open up browser windows without too much explanation and started to show HTTP headers and the information being passed between the web server and the user. Again with only a few words here and there he started to show how to turn off things like php signature on the webserver or change your scripts from .php extensions to like .html. Basically security through obscurity, something the previous 3 speakers all advocated against. Within the initial 5 minutes of his talk he already received a very excellent comment about this and the effectiveness of it. After a few more minutes of this mind numbing presentation I decided to leave. This may have been rude of me and maybe I didn’t give him enough of a chance, but in my opinion this guy did not belong at this conference. It was a little disappointing to have 1 hour of this conference wasted when someone like Christian could have used it talk more indepth.

So after spending an hour outside in the sun we came in to the hear the last speaker Chris Hubbard talk about Data Validation in PHP. Hubbard had a very well researched presentation and explained the many ways in which to validate input. He showed us several examples of validation, and the difference between simple checking (is it an integer?) and more complex checks (is this a valid email address?). He showed us examples of several third party applications that can be used for data validation and how they could be used. He also made it very clear that when using any type of third party application one should be cautious about trusting it and that you should really know what it does before you use it.

I really liked the way Chris Hubbard used real word examples of why data validation is so important and what can happen when you don’t validate data properly. He had a very strong presentation and obviously knew a lot about his subject matter. One thing that pissed me off during his presentation was that Tom (see above Stefan walking out) started to be very vocal and asked some pretty stupid questions. It was almost like he realized his presentation bombed and he wanted to sabotage Chris’s talk. At one point Chris was talking a bout regular expressions and Tom commented “That top one looks really good actually!”. What was funny is that Chris had just stated that the top regular expression was the one that was the worst of the three. Anyway these interruption only made me dislike this Tom guy even more.

After Chris Hubbard’s presentation the day was finished with an open forum with all of the speakers. I asked the first question about the best way to store credit card numbers if you really have to. After the expected “don’t” responses, I got some good answers on ways to protect the information by using mcrypt or even just storing the last 4 digits and possibly a hash of the credit card, or even just using a third party to handle that type of information. After the speakers brought up that a third party could be used a guy from Victoria become very vocal (very reminiscent of Tom earlier) about his company and what a good company they were. Basically his company handles the credit cards for you and thus you never have to store them. For the rest of the forum he continued to chime in about what his company was doing. Anyway after a few more questions about related issues the talk then swung to software patenting and the future. Bruce Perens was very bleak regarding the future, and really made it clear that action needs to be taken to fight these patents.

I am glad I attended this event. I was a bit disappointed however with some of the co-ordination and the additional costs of the conference in comparison with what was offered at the previous conference. I was also disappointed to hear that additional conferences held by OSEVENTS would no longer be PHP specific but rather open source events. While Vancouver has a need for open source events I find that there was a real opportunity and a solid niche to fill with PHP specific conferences. I don’t think it would be hard to come up with new topics or to fill events specifically on PHP. I hope this new direction by OSEVENTS will still maintain it’s core supporters (those from the PHP community) by having a heavy PHP contingent in upcoming conferences.

On Friday I went with a coworker to the PHP West Web Services conference being held at the Planetarium here in Vancouver. This conference had some very notable speakers from the PHP community such as Rasmus Lerdorf (Inventor of PHP). It is often hard to determine the usefulness of conferences, as they are often colossal wastes of time and money. However this conference was very practical and the speakers all had very informative talks.

To open PHP West was Rasmus Lerdorf with a talk titled Pragmatic PHP. While the talk itself had little to do with web services I thought it was probably the best of the day. It was refreshing to have someone give very practical ideas and solutions to very common problems as well as to promote the idea of simplicity. Rasmus stressed many times during his talk that you should not try to over think problems and that you should search for the easiest solution when possible. It was also fun to see him optimize a script as best as possible.

After Rasmus was Terry Chay talking about design patterns in PHP. Terry was full of energy and very enthusiastic as well as being quite practical. Reading out definitions like “Blah blah blah blah” and then putting it into relatable terms, “What that really means is … and can be applied by …”. It also looked like Terry put some major work into his presentation, which I must say, was totally kick ass.

After lunch was John Coggeshall and the first actual presentation on web services. He went through both consuming and creating web services. Maybe the reasoning behind having so little speakers on web services at a web services conference was the simplicity of using web services in PHP. John did a great job in showing how easy it is to consume a web service and even easier to setup a web service. One funny thing I noticed when John was talking was that he obviously had an American accent, which I must say made it quite funny when he would say ‘get variable’ as ‘giit variable’. J

Next up was Cal Henderson from Ludicorp the makers of the very popular (and local to Vancouver) Flickr. Cal’s talk was more on the development of Flickr and the role web services were given. It was interesting to hear some practical uses of web services. Cal pushed the message that by using web services you can give other people the ability to create new products for you that you otherwise might not have the time or resources to do. Cal presentation was pretty funny and he was very quotable. I think what made his presentation so damn funny (besides the British accent) was that he didn’t crack a smile once, even when he was making some very funny jokes. Anyway a good fun presentation from a local company in town!

Last presentation of the day was from Ilia Alshanetsky on XML in PHP. This presentation was probably the most technical of all of them, and very informative. However I think PHP West timed this presentation wrong, as having it at the end of the day was a bit much with all the information Ilia was presenting. Regardless I enjoyed the talk and thought it was pretty good overview on the different ways to handle XML in PHP.

To finish the conference off they brought on stage some blond promotions girl from Techvibes to give away free books and stuff from the promoters of the conference. I thought it was pretty funny how a person whose daily job is to promote things can barely read a persons name from a namecard. It was painful and a bit too much for us so we decided to leave (after getting our free books of course!).

All in all I was impressed by the conference and of PHP West for pulling this together. I now look forward to their next conference to come in June regarding PHP security.

Recently after doing an upgrade of PHP I noticed all my scripts to be breaking, and after doing a little bit of research I realized that they were breaking because I was calling my associative arrays (hashes) wrong. I was still using an old style in which you don’t put quotes around the key value of the array.

Thus an example of the bad way to do associative arrays is:

And the proper way is:

This used to work fine, however it looks like PHP is phasing this notation out (as they rightfully should) , thus in the latest version of PHP (4.3.10) you will start to see errors for such a notation.

Now you might say “what can I do?” you don’t want to have to go through all your code looking up associative arrays finding where you did the wrong notation. Thus I came up with a fancy pancy regular expression that will search and replace all associative arrays that have been written under the old notation and correct them. You can use this regex with sed if you like, that is what I did. Actually I wrote a script that went through all my directories recursively and corrected all my files. If you would like a copy of that just ask. Anyway without further delay here is the wonderful regular expression:

s/($([a-zA-Z_])*( *-> *[a-zA-Z_]*)*([))([a-zA-Z][a-zA-Z_]*)(])/1′5′6/g