Command line tool for eZ Publish, called "eep"

Mugo has a tool that we use internally to help with the main aspects of a developer's life: development, debugging and maintenance. The tool is called "eep", short for "Ease eZ Publish". It's a command line tool which provides many functions and allows for accomplishing tasks quickly by integrating with tools like awk, grep and xargs for powerful one-liners and supports rapid development of powerful bash scripts by leveraging eep as a library of eZ Publish specific operations.

The details

eep is a standalone script, however it can hook in any 4.x kernel in order to operate on a particular eZ instance.

eep is intended to be useful to developers; not content managers or what have you. It is a little dangerous and focuses on getting the job done.

 
> cd http/dbx
> eep use ezroot .
msg:: Reseting ezroot
>
> ## Danger! Danger!
> eep contentclass deleteclass folder
 

The output format is modeled after mysql output:

 
> eep contentnode info 8405
 
 +--------------------------+----------------------------------+
 I contentnode info [8405]                                     | 
 +--------------------------+----------------------------------+
 I                      key | value                            | 
 +--------------------------+----------------------------------+
 |                     Name | Frequently Asked Questions       | 
 |          ContentObjectID | 8503                             | 
 |               MainNodeID | 8405                             | 
 |          ClassIdentifier | downloadable_file                | 
 | PathIdentificationString | frequently_asked_questions       | 
 |               PathString | /1/2/8405/                       | 
 |             ParentNodeID | 2                                | 
 |          CurrentLanguage | eng-US                           | 
 |     ContentObjectVersion | 1                                | 
 |                 RemoteID | 68e90a5f7fe2370a73374da2a3435634 | 
 |                 IsHidden | 0                                | 
 |              IsInvisible | 0                                | 
 | ContentObjectIsPublished | 1                                | 
 |    Reverse related count | 1                                | 
 |           Children count | 0                                | 
 |                URL Alias | Frequently-Asked-Questions       | 
 +--------------------------+----------------------------------+
 

The basic concept of eep is to:

• provide a bunch of operations that you can do on an ezpublish instance
• focus on the ability to link operations together and use them with existing
• tools like grep, awk, xargs, sort, uniq, paste, wc, etc. to build up powerful one-liners
• support rapid development of shell scripts that would otherwise take hours to write and debug

The guiding values have thus far been:

• this is built for developers; it's a tool box for expert users
• don't duplicate functionalities available from other command line tools
• expect that the codebase exists as a catalog of PHP around the eZ Publish API
• make operations that focus on object and node IDs, which means that you can scale and multi-thread operations

eep is available on github at: https://github.com/mugoweb/eep

Installation is documented in install.txt and amounts to causing "eep" to simply invoke the script. Note that for linux installs, the command line completion stuff is very nice; even now with just basic functionality it's big help.

In addition to leveraging all the standard tools, making eep a command line tool provides a couple sweet benefits. The first is that spreading a heavy processing load across multiple CPUs is trivial. The second takes advantage of a custom daemon to schedule and control large lists of tasks; the daemon also provides operability via the eZ admin interface. We aren't releasing the daemon at this time; we are still rolling it into a production setting.

And now for some examples.

Example 1: Delete all unused content classes.

 
> eep list contentclasses 
 +----------------------------+----+------+----------------------------------+--------+------------------------------------+---------+
 I list content classes                                                                                                              | 
 +----------------------------+----+------+----------------------------------+--------+------------------------------------+---------+
 I                 Identifier | Id |    # |                         RemoteID |   Lang |                               Name |   Group | 
 +----------------------------+----+------+----------------------------------+--------+------------------------------------+---------+
 |               announcement | 59 |    2 | 4508764cae43f93a9a17480685168739 | eng-US |                       Announcement |     DBX | 
 |                    article | 16 |    0 | c15b600eb9198b1924063b5a68758232 | eng-US |                            Article |         | 
 |                asset_class | 57 |    5 | 851dade1c910ad26700b30a2c70ca4b2 | eng-US |                        Asset Class | Content | 
 |                     banner | 45 |   10 | 9cb558e25fd946246bbb32950c00228e | eng-US |                             Banner | Content | 
 |        common_ini_settings | 14 |    1 | ffedf2e73b1ea0c3e630e42e2db9c900 | eng-US |                Common ini settings |   Setup | 
 |               custom_error | 91 |    2 | fe2fd148254033388798af7b6218d35d | eng-US |                       Custom Error |     DBX | 
 |                 deployment | 97 |   11 | 6e686a81fb81d07e9ff535effa86391f | eng-US |                         Deployment | Content | 
 |                 disclaimer | 90 |   12 | 390e0f705816c632e56abd419c792e64 | eng-US |                         Disclaimer | Content | 
 |        product_disclosures | 73 |   11 | 4a1ed1df1cdb1846d6893a9b10c0687d | eng-US |                        Disclosures | Content | 
 |      product_distributions | 68 |   21 | 358c65fae841d33e28f4ee9476214058 | eng-US |                      Distributions | Content | 
 |          downloadable_file | 89 | 1393 | ceb50523639d1b2c1f5c76c817508ae6 | eng-US |                  Downloadable File |   Media | 
 |          product_downloads | 70 |   53 | b4f38188ebe5d44d624c12890f83a20c | eng-US |                          Downloads | Content | 
 |                   faq_item | 82 |   25 | b34c64ee3ee76964b25d8202eabd7e7f | eng-US |                           FAQ Item |     DBX | 
 |                       file | 28 |    1 | 637d58bfddf164627bdfd265733280a0 | eng-US |                               File |   Media | 
 |                      flash | 29 |    0 | 6cd17b98a41ee9355371a376e8868ee0 | eng-US |                              Flash |   Media | 
 |             flash_recorder | 30 |    0 | e349c947fd306299418be35b07b9a940 | eng-US |                     Flash recorder |   Media | 
 |                      focus | 56 |    8 | be8710cb34466d9460e207c35d8e951b | eng-US |                              Focus | Content | 
 |                     folder |  1 |  368 | a3d405b81be900468eb153d774f4f0d2 | eng-US |                             Folder | Content | 
 |              windows_media | 36 |    0 | 223dd2551e85b63b55a72d02363faab6 | eng-US |                      Windows media |   Media | 
 +----------------------------+----+------+----------------------------------+--------+------------------------------------+---------+
 

(note that the output is truncated for the blog post)

 
> eep list contentclasses | awk '$6==0'
 |                    article | 16 |    0 | c15b600eb9198b1924063b5a68758232 | eng-US |                            Article |         | 
 |                      flash | 29 |    0 | 6cd17b98a41ee9355371a376e8868ee0 | eng-US |                              Flash |   Media | 
 |             flash_recorder | 30 |    0 | e349c947fd306299418be35b07b9a940 | eng-US |                     Flash recorder |   Media | 
 |                    gallery | 38 |    0 | 6a320cdc3e274841b82fcd63a86f80d1 | eng-US |                            Gallery | Content | 
 |              global_layout | 32 |    0 | f0271811b913befa8f062527e909f15e | eng-US |                      Global layout | Content | 
 |                    infobox | 25 |    0 | 0b4e8accad5bec5ba2d430acb25c1ff6 | eng-US |                            Infobox | Content | 
 |                       link | 34 |    0 | 74ec6507063150bc813549b22534ad48 | eng-US |                               Link | Content | 
 |              multicalendar | 26 |    0 | 99aec4e5682414517ed929ecd969439f | eng-US |                      Multicalendar | Content | 
 |                       poll | 27 |    0 | 232937a3a2eacbbf24e2601aebe16522 | eng-US |                               Poll | Content | 
 |                  quicktime | 35 |    0 | 16d7b371979d6ba37894cc8dc306f38f | eng-US |                          Quicktime |   Media | 
 |                 real_video | 37 |    0 | dba67bc20a4301aa04cc74e411310dfc | eng-US |                         Real video |   Media | 
 |                silverlight | 47 |    0 | 8ab17aae77dd4f24b5a8e835784e96e7 | eng-US |                        Silverlight |   Media | 
 |               flash_player | 31 |    0 | 20b2ed0982343e6e0a550f7f0c137e06 | eng-US |                 Video/Flash Player |   Media | 
 |              windows_media | 36 |    0 | 223dd2551e85b63b55a72d02363faab6 | eng-US |                      Windows media |   Media | 
 

 
> eep list contentclasses | awk '$6==0 {print $2}'
article
flash
flash_recorder
gallery
global_layout
infobox
link
multicalendar
poll
quicktime
real_video
silverlight
flash_player
windows_media
 

 
> eep list contentclasses | awk '$6==0 {print $2}' | xargs -IOID eep contentclass deleteclass OID
Deleting 0 objects.
Done deleting objects.
PHP Fatal error:  Call to a member function isClassAttributeRemovable() on a non-object in /home/dfp/http/dbx/kernel/classes/ezcontentclass.php on line 815
PHP Stack trace:
PHP   1. {main}() /home/dfp/eep/eep.php:0
PHP   2. require_once() /home/dfp/eep/eep.php:130
PHP   3. contentclass_commands->run() /home/dfp/eep/modules/contentclass/index.php:311
PHP   4. contentclass_commands->deleteClass() /home/dfp/eep/modules/contentclass/index.php:256
PHP   5. eZContentClass->remove() /home/dfp/eep/modules/contentclass/index.php:145
PHP   6. eZContentClass->isRemovable() /home/dfp/http/dbx/kernel/classes/ezcontentclass.php:758
PHP   7. eZContentClass->removableInformation() /home/dfp/http/dbx/kernel/classes/ezcontentclass.php:774
Fatal error: eZ Publish did not finish its request
The execution of eZ Publish was abruptly ended, the debug output is present below.
xargs: eep: exited with status 255; aborting
 

Well, that's annoying. I wanted to remove all the unused content classes; but it appears that the first useless content class has a problem. My guess is that it's because the "article" class makes use of an attribute that is not actually in the system.

 
> eep list attributes article
 +-----------------+--------+-----+------------+------+------+------+-----+-----------------+
 I list attributes of class: article                                                        | 
 +-----------------+--------+-----+------------+------+------+------+-----+-----------------+
 I      Identifier |   Lang |  Id |       Type | Srch | Reqd | Info | Pos |            Name | 
 +-----------------+--------+-----+------------+------+------+------+-----+-----------------+
 |           title | eng-US | 183 |   ezstring |    1 |    1 |    0 |   1 |           Title | 
 |     short_title | eng-US | 184 |   ezstring |    1 |    0 |    0 |   2 |     Short title | 
 |          author | eng-US | 185 |   ezauthor |    0 |    0 |    0 |   3 |          Author | 
 |           intro | eng-US | 186 |  ezxmltext |    1 |    1 |    0 |   4 |         Summary | 
 |            body | eng-US | 187 |  ezxmltext |    1 |    0 |    0 |   5 |            Body | 
 | enable_comments | eng-US | 188 |  ezboolean |    0 |    0 |    0 |   6 | Enable comments | 
 |           image | eng-US | 189 |    ezimage |    0 |    0 |    0 |   7 |           Image | 
 |         caption | eng-US | 190 |  ezxmltext |    1 |    0 |    0 |   8 | Caption (Image) | 
 |    publish_date | eng-US | 191 | ezdatetime |    1 |    0 |    0 |   9 |    Publish date | 
 |  unpublish_date | eng-US | 192 | ezdatetime |    1 |    0 |    0 |  10 |  Unpublish date | 
 |            tags | eng-US | 193 |  ezkeyword |    1 |    0 |    0 |  11 |            Tags | 
 |     star_rating | eng-US | 194 | ezsrrating |    0 |    0 |    0 |  12 |     Star Rating | 
 +-----------------+--------+-----+------------+------+------+------+-----+-----------------+
 

It could only be 'star rating'. What is the status of the extension:

 
> eep list extensions 
 +--------------------+--------------------+----------------+---------------+---------------+
 I list extensions                                                                          | 
 +--------------------+--------------------+----------------+---------------+---------------+
 I            folders |   ActiveExtensions | A.A.Extensions |        design |       modules | 
 +--------------------+--------------------+----------------+---------------+---------------+
 |  contentdeployment |                    |                |               |               | 
 |           ezjscore |           ezjscore |                |      ezjscore |      ezjscore | 
 |              ezodf |                    |                |               |               | 
 |         ezcomments |                    |                |               |               | 
 |    ezscriptmonitor |                    |                |               |               | 
 |               ezsi |                    |                |               |               | 
 |               ezie |                    |                |               |               | 
 |        data_import |                    |                |               |               | 
 |      ezmultiupload |      ezmultiupload |                | ezmultiupload | ezmultiupload | 
 |       ezstarrating |                    |                |               |               | 
 |                dbx |                dbx |                |           dbx |           dbx | 
 |             ezfind |             ezfind |                |        ezfind |        ezfind | 
 |            ezwebin |            ezwebin |                |       ezwebin |               | 
 |             ezflow |             ezflow |                |        ezflow |        ezflow | 
 |           ezmbpaex |                    |                |               |               | 
 |        ezformtoken |                    |                |               |               | 
 |               ezoe |               ezoe |                |          ezoe |          ezoe | 
 |               ezwt |                    |                |               |               | 
 | ezprestapiprovider | ezprestapiprovider |                |               |               | 
 |             ezless |                    |                |               |               | 
 |     ezgmaplocation |                    |                |               |               | 
 |            rautils |                    |                |               |               | 
 +--------------------+--------------------+----------------+---------------+---------------+
 

So, the "ezstarrating" extension is not enabled. Let me hack that into settings/override/site.ini.append.php.

 
> eep list extensions | grep star
 |       ezstarrating |       ezstarrating |                |  ezstarrating |               | 
 

And then try to delete the content class:

 
> eep contentclass deleteclass article
Deleting 0 objects.
Done deleting objects.
 

and then delete all the useless content classes again, and confirm:

 
> eep list contentclasses | awk '$6==0 {print $2}' | wc -l
0
 

Ok, ok; using wc was a little bit unnecessary, but it did generate a nice "0".

Example 2: Handle biggish installations

 
> eep list contentclasses 
 +---------------------+----+--------+----------------------------------+--------+---------------------+-----------+
 I list content classes                                                                                            | 
 +---------------------+----+--------+----------------------------------+--------+---------------------+-----------+
 I          Identifier | Id |      # |                         RemoteID |   Lang |                Name |     Group | 
 +---------------------+----+--------+----------------------------------+--------+---------------------+-----------+
 | onix_bisac_category | 55 |   2733 | 85d41773c35f29bcbe41aea2eaec93c9 | eng-CA |      BISAC category | Bookshelf | 
 |                blog | 19 |      1 | 3a6f9c1f075b3bf49d7345576b196fe8 | eng-CA |                Blog |   Content | 
 |           blog_post | 20 |      6 | 7ecb961056b7cbb30f22a91357e0a007 | eng-CA |           Blog post |   Content | 
 |     book_annotation | 71 |    671 | f29e51311e8aa9821ef98e78f6c00a4b | eng-CA |     Book Annotation | Bookshelf | 
 |         book_folder | 49 |     27 | acaee61d5596130a584cae3d91ee53d3 | eng-CA |         Book Folder | Bookshelf | 
 | common_ini_settings | 14 |      1 | ffedf2e73b1ea0c3e630e42e2db9c900 | eng-CA | Common ini settings |     Setup | 
 |  contributor_folder | 50 |     26 | 80ec2434df8e66d72baa99f1a1079733 | eng-CA |  Contributor Folder | Bookshelf | 
 |  documentation_page | 24 |     12 | d4a05eed0402e4d70fedfda2023f1aa2 | eng-CA |  Documentation page |   Content | 
 |                file | 28 |      4 | 637d58bfddf164627bdfd265733280a0 | eng-CA |                File |     Media | 
 |      flash_recorder | 30 |      1 | e349c947fd306299418be35b07b9a940 | eng-CA |      Flash recorder |     Media | 
 |              folder |  1 |    302 | a3d405b81be900468eb153d774f4f0d2 | eng-CA |              Folder |   Content | 
 |           frontpage | 23 |      6 | e36c458e3e4a81298a0945f53a2c81f4 | eng-CA |           Frontpage |   Content | 
 |          game_cover | 73 |     48 | 3101958361cf3b76b9c804ccc0aab1b8 | eng-CA |          Game Cover | Bookshelf | 
 |            giveaway | 76 |     34 | 00edae306718c75cabbfcb738a6c0df1 | eng-CA |            Giveaway | Bookshelf | 
 |       global_layout | 32 |      1 | f0271811b913befa8f062527e909f15e | eng-CA |       Global layout |   Content | 
 |               image | 33 |    918 | f6df12aa74e36230eb675f364fccd25a | eng-CA |               Image |     Media | 
 |                list | 65 |    332 | faa59e289d6d7fb6636ea2d33fc92f41 | eng-CA |                List | Bookshelf | 
 | list_bisac_category | 75 |    851 | 6ca9b6c4ac241ff77d2f9cfe0566c17c | eng-CA | List BISAC category | Bookshelf | 
 |             message | 67 |     34 | 1db77616a9436fdb19d8d836c66dd8db | eng-CA |             Message | Bookshelf | 
 |            otp_post | 64 |    178 | 1bd7e59a75b52e276b0755aa62e089db | eng-CA |   Off The Page post | Bookshelf | 
 |    onix_contributor | 54 |  27524 | 6885e3e3d924dd57e12019a56a7fa716 | eng-CA |    ONIX Contributor | Bookshelf | 
 |      onix_mediafile | 56 |  37415 | 658867ac3bd41a95e71eee070b0542e5 | eng-CA |      ONIX MediaFile | Bookshelf | 
 |      onix_othertext | 57 | 134714 | 9daf9eb9ff4f79f51ac58bd8441f3562 | eng-CA |      ONIX OtherText | Bookshelf | 
 |          onix_prize | 62 |   9481 | d37435393a5d53231e188edeb1b93f1d | eng-CA |          ONIX Prize | Bookshelf | 
 |        onix_product | 52 |  52364 | fdbb7e3150f22c72f5fadb8963ffc045 | eng-CA |        ONIX Product | Bookshelf | 
 |      onix_publisher | 53 |    745 | d0287045337b0f2fc9c9e03d188f23e6 | eng-CA |      ONIX Publisher | Bookshelf | 
 |         onix_series | 61 |   1961 | d492b9b7a14b8e0e64ad129270f67f7c | eng-CA |         ONIX Series | Bookshelf | 
 |   onix_supplydetail | 59 |  53704 | f91869f0f114a08b8b3a1842709fc665 | eng-CA |  ONIX Supply Detail | Bookshelf | 
 |        onix_website | 58 |   4841 | b5acae96ec526d0bdb0c0d7996b1ffd7 | eng-CA |        ONIX Website | Bookshelf | 
 |    publisher_folder | 51 |     27 | e7f78d2f9c8a7bb10d5afc57092dc0c7 | eng-CA |    Publisher Folder | Bookshelf | 
 |      report_message | 72 |    274 | 64eeb2569da631066cb024b85a812af2 | eng-CA |      Report Message | Bookshelf | 
 |              review | 70 |     95 | 3331aaebb0ca37e1834d0bcc3ccabfcf | eng-CA |              Review | Bookshelf | 
 |       review_folder | 69 |     86 | 63b33166599d4b499b62ac76b12a89c5 | eng-CA |       Review Folder | Bookshelf | 
 |       series_folder | 60 |     27 | 5b140326284981ede98b9bb6d1ed3cba | eng-CA |       Series Folder | Bookshelf | 
 |      simple_content | 74 |      2 | 690aa81fdb1ca8b18a4f84a37f2e2ccb | eng-CA |      Simple content | Bookshelf | 
 |       template_look | 15 |      1 | 59b43cd9feaaf0e45ac974fb4bbd3f92 | eng-CA |       Template look |     Setup | 
 |                user |  4 |   1398 | 40faa822edc579b02c25f6bb7beec3ad | eng-CA |                User |     Users | 
 |    user_affiliation | 63 |      4 | 91f4a128ca662425698c314ac38fc1b0 | eng-CA |    User Affiliation | Bookshelf | 
 |          user_group |  3 |      6 | 25b4268cdcd01921b808a0d854b877ef | eng-CA |          User group |     Users | 
 +---------------------+----+--------+----------------------------------+--------+---------------------+-----------+
 

That's 134714 instances of onix_othertext. Handling this many instances in PHP is something of a problem because memory leaks will crash your script well before you've handled all those objects.

 
> eep contentclass fetchallinstances onix_othertext
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 71 bytes) in /home/dfp/eep/lib/eepHelpers.php on line 352
PHP Stack trace:
PHP   1. {main}() /home/dfp/eep/eep.php:0
PHP   2. require_once() /home/dfp/eep/eep.php:130
PHP   3. contentclass_commands->run() /home/dfp/eep/modules/contentclass/index.php:311
PHP   4. contentclass_commands->fetchallinstances() /home/dfp/eep/modules/contentclass/index.php:265
PHP   5. eep::displayNonObjectList() /home/dfp/eep/modules/contentclass/index.php:171
Fatal error: eZ Publish did not finish its request
The execution of eZ Publish was abruptly ended, the debug output is present below.
 

So eep supports limit and offset:

 
> eep contentclass fetchallinstances onix_othertext --limit=100000 --offset=40000
 +---------+-----+---+----------------------------------+---------------------------------------------+
 I All instances of content class 'onix_othertext'                                                    | 
 +---------+-----+---+----------------------------------+---------------------------------------------+
 I  Object | Sid | V |                        Remote Id |                                        Name | 
 +---------+-----+---+----------------------------------+---------------------------------------------+
 | 3044426 |   1 | 1 | b5a96b469819f9f752bf8f4118ff25ba |                            Main description | 
 | 3044427 |   1 | 1 | b67005432cac1b0afd231f60c1a6ba2c |                                 Review text | 
 | 3044431 |   1 | 1 | 6e57bd7c41b752e9bca9b085ce0d1281 |                            Main description | 
 | 3044435 |   1 | 1 | 8e88848541395e344384f2b65e67f3e0 |                            Main description | 
 | 3044439 |   1 | 1 | de8550226af4248bd951a775acee4545 |                            Main description | 
 

... and so on, generating 100,000 rows of data.

Example 3: Managing data

eep wraps the lovely toString() and fromString() API functions. This is the built-in 'help' output:

 
> eep attribute
Available commands:: help, delete, fromstring, tostring, migrate, newattributexml, update delete
 
- deletes an attribute from class and objects
  eep use ezroot <path>
  eep use contentclass <class identifier>
  eep attribute delete <attribute identifier>
  
fromstring
- calls FromString() on the attribute
  eep use contentobject <object id>
  eep attribute fromstring <attribute identifier> <new value>
  or
  eep attribute fromstring <content object id> <attribute identifier> <new value>
  
tostring
- calls ToString on the attribute
  eep use contentobject  <object id>
  eep attribute tostring <attribute identifier>
  or
  eep attribute tostring <content object id> <attribute identifier> 
 
migrate
- copies data from one attribute to another within a content class
- todo, report available conversions
  currently supported are "rot13" for testing and "time2integer"
  eep use ezroot <path>
  eep use contentclass <class identifier>
  eep attribute migrate <src attribute> <conversion> <dest attribute>
                  
newattributexml
- dumps xml that can be edited and then imported
  eep attribute newattributexml
 
update
- updates objects with new attribute and also the class; will resume after a  partial update
  eep use ezroot <path>
  eep use contentclass <class identifier>
  eep attribute update <path to newattributexml file>
 

Download and contribute to eep on GitHub: https://github.com/mugoweb/eep

Read a case study on solving a client problem with eep.