Excluding fields in the mongodb/mongodb library

Leave a comment

I am using the mongodb/mongodb library for a project of mine.  The API seems fairly different from the old PECL library and it also came with some other, albeit unexpected, baggage.

My understanding of the library is that it is intended to be a replacement for the existing PECL library and that it intends to strike a balance between the core library functionality while also giving it a more agile, for lack of a better term, release cycle.  What that means is that there is some basic MongoDB functionality written in C with the forward-facing API being written in PHP.

In and of itself this is not an issue… except for the one thing I was trying to do.  One of the practices I’ve heard about Mongo is to get Mongo to do as much as it can, but not to worry too much about complicated joins and such as you would in SQL.  In other words, don’t shy away from bringing data into the application to do some processing.

That was the practice I followed, which worked fine up until my data size started to increase.  Then I ran into an issue where I was seeing 50% of my response time being eating up by BSONDocument and BSONArray unserialization calls.  This was due to a sub-document I had in the document that could actually be quite large.

“Hmm,” I thought to myself.  “I’m sure glad I didn’t see this a month from now.”  I did some research and waffled a little between using MapReduce or using the newish aggregation features.  I opted for the aggregation feature, which is so incredibly powerful.  I see myself misusing this quite often.

But after changing the code to make it work with aggregation I still had the problem of the unserialization.  The documents are returned as a whole.  In the core functionality you can exclude individual fields by doing something like this:

The second parameter (classification.category) tells Mongo to omit that field in the result set.  But I couldn’t find a way to mimic this behavior in the new library.  So I opened an issue on GitHub (when you run into a problem, contact the library maintainers or post on SO.  Don’t just leave it) asking how I might run this.

Jeremy Mikola responded with the solution.  There is a second option that you can put in that is documented, but I don’t think it is directly clear from the documentation exactly what it does.  Or, at least, the description isn’t very SEO friendly (Google-Based-Development FTW!).

How does that look?

Like this:

BOOM! 7.5 seconds off the response wall clock time.

Micro-Webinar: Using the Instruction Navigator in Magium – March 23, 2016: 12:00 EDT

Leave a comment

On Wednesday, March 23, 2016 I will be hosting a 15 minute micro-webinar on how to use the instruction navigator.

The instruction navigator is used to navigate around your Magento site when you need a series of clicks and/or mouse moves. A good example of this is navigating to the customer login screen.

This 15 minute webinar will go over the instruction navigator and how you can use it to make complex test instructions easily repeatable.

Webinar Location

Powered by WPeMatico

Command line features in Magium

Leave a comment

While the rest of you were on Spring Break I was , well, too. But I also managed to finegle some time in between building snow forts and violent illness to add some new features to Magium.

CLI tooling is fairly popular in PHP-land and given that I want to be popular too I figured it would be a good idea to add some CLI tooling to Magium. Some of this is in preparation for some new tooling I will be adding to the magiumlib.com website in the next few weeks.

The CLI tool is found in vendor/bin/magium. If you have MagiumMagento installed, as of the time of this writing, these are the options you will see:

config:set Modifies a setting
config:unset Removes a setting
element:get Retrieves the default values for a configurable element
element:list Extracts all of the AbstractConfigurableElement-based objects
element:set Modifies a property value for a configurable element
element:unset Removes an abstract configurable element’s property
magento:extract-navigation-xpath Will run a series of tests on the provided URL to attempt to extract the navigation Xpath for the theme configuration
magium:init Creates the magium.json config file in the specified project root directory
magium:webdriver Configures the WebDriver settings

Magium Commands (init|webdriver)

Before running anything you need to initialize the environment. That basically means writing the magium.json file. Run:

And you’re set.

The webdriver command allows you to make some basic changes to the webdriver settings. More functionality will be expanded upon later.

You can set the webdriver URL

You can very easily change the browser

Config Commands (set|unset)

These are not currently used but will be used with some of the SaaS features that I am working on. They will allow you to set various arbitrary project settings, though most will actually use commands in the element namespace.

Element Commands (get|list|set|unset)

These are used to set configuration options for elements that extend MagiumAbstractConfigurableElement. In other words, they have been designed on the class level to be configurable. Some examples of this are identities and themes.

Say that you want to change the default email address for your customer identity. You can create the /configuration/Magium/Magento/Identities/Customer.php file and define it there or you can execute the following series of commands.

First, find the email address property using element:get (the format is the class name followed by the (optional) stripos-based filter

Now you know the email address property is called emailAddresss.

Next up is to set the new value.


Not sure what your options are? Try element:list.

You can also make it copy and paste friendly (since CLI’s don’t know that the is the namespace separator).

Magento Commands (extract-navigation-xpath)

Currently there is only one command for the Magento namespace, but others will be added. Extract-navigation-xpath can be used to help figure out what is the path to the navigation components are.

Powered by WPeMatico

How to start Selenium testing on Magento (or, Succeeding at Selenium in under 10 minutes)

1 Comment

I had some good talks with some good folks this past week. Now entering the 4th month of working on Magium I can honestly say that this past week was the most exciting week I’ve had. It was a week when I allowed myself to think “maybe I am right about this after all!”

But, truth be told, it’s been a struggle to gain some traction. People know, and I mean really know, that they should be automating this part of testing. Every single person who I’ve talked about testing via the browser has said almost the exact same thing. It’s a great idea, but it takes too long and is too tedious.

Browser testing fails, in my humble opinion, because of problems that almost all software has. It is a very common problem. And it’s so common I’m going to put it in <H1> tags to make sure everyone sees it.

Software should have easy, early wins

In other words,with software, especially new or unfamiliar software, you should have something useful to show with a minimal amount of effort. That’s why, I believe, despite there being several good packages out there for testing Magento with Selenium, including good ones by Magento, I believe that the reason very few use them is because they are not able to “ease” into them.

So what I am going to do is show you how you can get started with Magium, testing Magento using Selenium, in under 10 minutes.

These quick wins are on me :-)

Step 1: Download and install Selenium Server and Chromedriver

Download Selenium Server and Chrome WebDriver

Start it as so, replacing the jar file with whatever the current version is and the chromedriver.exe with whichever version you downloaded:

Step 2: Download and install Magento CE 1.9 with the Sample Data

Note: this doesn’t count against the 10 minutes

Make sure that it is working first before you start the timer. (I presume you don’t need instructions on how to do this)

Step 3: Clone the sample test cases

Step 4 – Edit 4 files

Edit the following files:

  • configuration/Magium/Magento/Identities/Admin.php – Enter your admin credentials
  • configuration/Magium/Magento/Identities/Customer.php – Enter some default user credentials
  • configuration/Magium/Magento/Themes/Admin/ThemeConfiguration.php – Change the baseUrl setting
  • configuration/Magium/Magento/Themes/Magento19/ThemeConfiguration.php – Change the baseUrl setting

Step 5 – Run composer

or if you have a custom phar installed

Step 6 – Run PHPUnit

If you are using PHPStorm all you need to do is right click on phpunit.xml.dist and select ‘Run phpunit.xml.dist’

If you are running PHPUnit from the command line you will need to exectue

And with that you should see 15 tests running and passed after a few minutes.


Powered by WPeMatico

The Basics Of The Instruction Navigator

Leave a comment

Instruction Navigator

The instruction navigator is similar to the BaseMenu navigator in its intent, and some of the underlying functionality. However, there is a crucial difference in that the base menu navigator uses iteration for predictable elements. The instruction navigator is for non-iterative navigation instructions.

It is also intended for general functionality that might change based off of your theme. Unlike the SystemConfiguration navigator, which is complicated but prone to stasis, something like logging in to your account is both complex and prone to change. It involves clicks on non-pattern-able HTML elements and it is heavily reused. A lot of navigation in Magento is going to be directly scripted via WebDriver when it is not repeatable and not configurable. But if both of those conditions are true (repeatability and configurability) then that makes an action a candidate for using the instruction navigator.

The instruction set is a multi-dimensional array in the form of

Note that the individual instruction arrays are not key => value pairs.

Currently there are two kinds of actions

  1. Mouse Move
  2. Mouse Click

There is no particular reason other than utility for those being the only actions being available.

The individual actions are the string values mouseClick, and mouseMoveTo. They are generally going to be referenced by the static constants in the MagiumWebDriverWebDriver class. The ThemeConfiguration class provides a simple example for navigating to the login page.

protected $loginInstructions = [
[MagiumWebDriverWebDriver::INSTRUCTION_MOUSE_CLICK, '//div[@class="account-cart-wrapper"]/descendant::span[.="Account"]'],
[MagiumWebDriverWebDriver::INSTRUCTION_MOUSE_CLICK, '//div[@id="header-account"]/descendant::a[@title="My Account"]']

That’s about it.

Powered by WPeMatico

How the BaseMenu navigator works

Leave a comment

How The BaseMenu Navigator Works

The BaseMenu navigator class provides functionality for navigating over simple, hierarchical HTML structures such as a frontend nested menu. It starts with a base Xpath that is used to note the root of element, usually based off of an HTML ID, followed by iteratively executed compounded child Xpath statements.

To see how this works consider the following HTML

The root Xpath would be //ul[@id="nav"]. The child Xpath would be something like li[.="%s"] which would be concatenated onto the root with the “descendant” instruction. Then you would provide ‘Child 1/Child 2’ to the navigator and it would execute the following commands

  1. Mouse move to element //ul[@id="nav"]/descendant::li[.="Child 1"]
  2. Mouse move to element //ul[@id="nav"]/descendant::li[.="Child 1"]/descendant::li[.="Child 2"]
  3. Click on element //ul[@id="nav"]/descendant::li[.="Child 1"]/descendant::li[.="Child 2"]

Note that this is a simple example and wouldn’t actually work because the text for //ul[@id="nav"]/descendant::li[.="Child 1"] actually contains some spaces and so the exact text match will not work. That’s why the theme’s child node selector Xpath is the more complicated a[concat(" ",normalize-space(.)," ") = " %s "]/...

As noted in other pages, if you can use the base functionality in the BaseMenu navigator the preferred method of customization is not to customize the navigator but to customize the theme through the use of the functionality in AbstractConfigurableElement. The customized child Xpath will need to have an sprint() place holder for the string. In the Magento 1.9 theme each menu item will have an element with the text held inside a parent element that has the depth specified as a CSS class. Your customization will need to include the same. But if you are using the base Magento theme, and reusing the core menu functionality and only changing the skin, no customizations should be necessary.

Using the BaseMenu navigator within the test is quite easy. The AbstractMagentoTestCase class contains a helper function call getNavigator(). This method will construct a class name and request it from the DIC.

Powered by WPeMatico