My concerns are specifically in regards to Categories and the goals it is attempting to achieve as its mechanics have become quite different.
From the category database table, I have noticed the value column, data[1-5] and data_domain[1-5], and security_domain columns have been eliminated. I can see that data and data_domain columns have now been implemented as Attribute objects via the __ATTRIBUTES__ property name.
In regards to the value column, it appears this can/may be implemented as an Attribute, but requires more coding than previous (RC1) Category implentation. After development of a couple home-made modules using RC1 that use PN Categories, I am having issues making them work as before. The major issue is with static stata.
Use of static data
It appears Categories is not used for static data (e.g. Yes, No, Maybe) types as I have not seen any module implement this. For example, News module could use static data types for its pending_status column, but it does not. It does appear that a record or obect can belong to one or more categories and this is how Categories is now used. From this, it seems that one of PN's goals is to categorize data objects in an organized fashion.
As I look at the Category code to determine how to use static data, I determined I can use values from Category attributes in a drop down from the Smarty plugin pnRender/plugins/function.selector_category.php. I can save this value and then render it correctly using the same plugin. Now, when I view the list of records with static data as attributes, I can not seem to make this work with the existing Category code. This would require knowledge of the parent category and then to look up the child who has an attribute stored as the value I need.
e.g. ($object = some object, $cat = Category object)
$object['PRODUCT_TYPE'] = 'WIDGET2';
$cat['display_name'] = 'The Widget Number Two';
$cat['__ATTRIBUTES__']['code'] = 'WIDGET2';
In order to find the category whose attribute 'code' has value 'WIDGET2', I must first know the Parent Category, then get all children categories, then examine each child category's attribute of 'code' and check for value of 'WIDGET2'. Once I know this, then I can display the Category's display name 'The Widget Number Two'. This seems almost overkill which brings me to ask, is this a correct usage for Categories as static data?
The need for static data is a major concern as I use static data everywhere. I also store static data 'values' in the database and not the Category ID. This is necessary especially when data is imported from another source. The data could include static data values that can then be mapped to a category. If I have 100,000 records, I don't want to look up every category to find its ID and then store it as a category object. If I have 10 fields per object that use static data, that turns into 100,000 records x 10 fields = 1,000,000 category records.
I am hoping that Robert Gasch and other devs who have put so much time into this component can speak to these concerns as I am not sure how I should implement at this point.
Thanks,
--Joe
Watch
GitHub Core
Show your support for Zikula! Sign up at Github account and watch the Core project!
GitHub Modules
Forum Activity
- michiel responded to »password problem« 10:01 AM
- mazdev responded to »Hide "Register new account" and change template to 3 col« 07:50 AM
- mesteele101 created topic »Zikula 1.3.3 - Site Search 1.5.2 - Unable to turn off plug-ins« 07:48 AM
- mesteele101 responded to »ERR (3): E_USER_ERROR: Smarty error: [in pagesvar:pagesitem2en line XXX]…« 25. May
- mazdev responded to »Pages 2.5.0 and updating - Page not found« 25. May
- mesteele101 responded to »Zikula 1.3.3 - Selecting a category in Pages not working« 25. May
- mdee created topic »How to implement returnpage ?« 25. May
Zikula Blog
- Anatomy of Open Source Projects on Mar 07
- Continuous Review on Mar 01
- Not Invented Here on Feb 24
- How to Contribute Your Code at Github on Jan 13
- 10 Steps to Coding-Nirvana: Tips for Successful Module Writing on Nov 12
- Submitting Bug Report Tickets That Get Results on Aug 17
- Cozi Tricks #1: Syntax Highlighting on Aug 07
Login
Categories as Static Data
-
- Rank: Registered User
- Registered: Jun 18, 2005
- Last visit: Jan 29, 2008
- Posts: 9
Posted: Jan 25, 2008 - 06:31 AM
-
- Rank: Registered User
- Registered: Jun 18, 2005
- Last visit: Jan 29, 2008
- Posts: 9
Posted: Jan 28, 2008 - 02:31 PM
Since I have not heard from anyone, I have some ideas to chew on...
My options (and maybe others) are to:
1) Create a StaticData module to be one table with id, parent_id, module, lang, name, value (code). This would be very similar to Categories, but would allow for different goals than Categories. But, it is soo similar :(
2) Deal with the attributes and perform the require code to first find the parent, then find the children, and then search the children for the code I am looking for.
3) Ask Robert Gasch to add back the value column in the cateogries table or rename the once 'value' column to 'code' and use it strictly as a 'code' or 'key' that is a unique child to its parent. Basically, if the parent_id is 1, then no other child directly or indirectly should have the same 'code'. This would enforce a unique category 'code' that would allow the category to also be used as static data.
e.g. $id = 1, $parent_id = null, $code = null
.. $id = 2, $parent_id = 1, $code = 'Y', $name = 'Yes'
.. $id = 3, $parent_id = 1, $code = 'N', $name = 'No'
.. $id = 4, $parent_id = 1, $code = 'M', $name = 'Maybe'
....
.... $id = 10, $parent_id = 4, $code = 'M1', $name = 'Maybe Level 1'
// ERROR - same code 'M' as in parent - Do not store !!!
.... $id = 11, $parent_id = 4, $code = 'M', $name = 'Maybe Level 2'
Any collaboration would be appreciated..
Thanks,
--Joe -
- Rank: Team Member
- Registered: Jan 05, 2003
- Last visit: May 28, 2010
- Posts: 775
Posted: Jan 29, 2008 - 12:41 AM
Hi,
we are currently discussing your situation on the coredevs list. Your arguments are basically valid, but just want to make sure that we don't break the consistency of the categories module.
You do understand that if you wish to use the 'value' value, that you won't be able to use the PN built-in categorization services (since that uses IDs) but will have to add respective columns to your data tables to store the static/category value?
Please confirm that my understanding of your issue is correct and that you understand the limitations of the approach as outlined above and we'll see what we can do for you.
Greetings
R -
Unregistered
- Registered: Mar 16, 2002
- Last visit: Oct 21, 2009
- Posts: 4294967287
Posted: Jan 29, 2008 - 06:52 AM
Hello Robert,
Thank you so much for responding and understanding my argument. In response to your question about how I may be unable to use the built-in categorization services, I may be confused.
I assume you will add back the 'value' column to the categories table. Once you do that, the category object or record will look as it does now, except for the addition of the 'value' column. I am thinking that the actual 'value' is basically another field that can be used as a unique code. This code value is unique to its immediate parent category and hence can be used as a lookup into the categories table given the parent ID or parent Path and the give code value. These two pieces of data would be needed to display the category's display name and may be used as criteria to join to a data object ( I will post this need and some ideas next ).
Therefore, when a category is added to the database and the 'value' property is added, then the cateogory must validate that any of its siblings can not have the same value (making the sibling (category) unique via parent and value). This immediate parent and child validation should be sufficient for unambiguos static data records.
e.g.
cat path
/a/b
/a/b/Yes (value = 'Y')
/a/b/No (value = 'N')
/a/b/Maybe (value = 'M')
/a/b/Mightof (value = 'M') // wrong - no duplicate value under same parent
/a/c
/a/c/Yes (value = 'Y') // ok - different parent
/a/c/No (value = 'N') // ok - different parent
/a/c/Maybe (value = 'M') // ok - different parent
** I did mention the original parent or root parent from the previous post, but that may be difficult. Therefore, immediate parent validation and unique child value should be fine (at least for my current needs).
With this, I don't see the Categories implementation being any different. I expect my 'edit' pages I could use dropdowns as before with the display_name as the HTML and the dropdown values using the 'value' column.
The 'display' and 'view' pages would require joining of a Category's parent ID or parent Path and the 'value' to the given data object's static value. My join (objJoin) post is next..
Thanks,
--Joe -
Unregistered
- Registered: Mar 16, 2002
- Last visit: Oct 21, 2009
- Posts: 4294967287
Posted: Jan 29, 2008 - 08:33 AM
This post is in regards to left joining data object with Categories as static data. As of now it is not easily possible to left join one data object with another given a static criteria (parent id or parent path) and field value (value). The need for this is rooted in performance. As of RC1, there was a Smarty plugin to display a category based on a value.
This was fine and worked well, but I knew it was not optimal and needed to be implemented better. Since I store my static values in the database, I do not have an associated Category ID to simply left join to a Category record). Therefore, in my 'display' and 'view' screens I had to perform a lookup on every static value. As far as performance goes, this is suboptimal as every static data field was looked up via the Smarty plugin and underlaying Category code. If we count the lookups on a 'view' page showing 30 records per page with 10 static values per page, this would require at least 301 database queries which also includes the main select query for the 30 records. That's not good, right?
Since I also use PNObjects (which is a very nice implementation BTW), I could not find a way to join the Category values to the actual Category. In this situation, it would be nice to join to another object given the parent id and category value. Since this information would be in every record, this is very possible. This would require an addition to the PN Object join code.
Below, are two _objJoin[] structures, the first is the current one field join. The second is a (possible) modified _objJoin[] structure with needed elements to join categories by value and parent_id.
Code
// THIS IS THE ONLY WAY TODAY TO JOIN VIA PN OBJECT
// Given the current _objType set to 'autos_imported', I want to join the category id from as below
$usrTblJoin = array ('join_table' => 'categories',
'join_field' => array('name'), // join categories.name
'object_field_name' => array('Name'), // 'Name' in autos_imported
'compare_field_table' => 'leased_cat_id', // field from autos_imported
'compare_field_join' => 'id'); // field from categories
// This would create the Left SQL join:
// autos_imported LEFT JOIN categories ON (autos_imported.leased_cat_id = categories.id)
// THIS IS THE ONLY WAY TODAY TO JOIN VIA PN OBJECT CODE
For static data, a join can be implemented if we know the parent_id and value (which we do). The _objJoin[] structure would require an addition piece of info to be used as a multi-value key.
Code
// THIS WOULD PROBABLY BE SUFFICIENT TO JOIN STATIC DATA - GIVEN PARENT_ID AND VALUE **
$parent_id = 7; // predetermined value from path lookup, etc.
$usrTblJoin = array ('join_table' => 'categories',
'join_field' => array('name'), // join categories.name
'object_field_name' => array('Name'), // 'Name' in autos_imported
'compare_field_table' => array(
array(type => 'value', value => $parent_id),
array(type => 'field', value => 'leased_cat_value')
), // the $parent_id is a value and leased_cat_value
// is a field from autos_imported
'compare_field_join' => array(
array(type => 'field', value => 'parent_id'),
array(type => 'field', value => 'value')
) // the multi-value key fields parent_id and value from categories
// This could create the Left SQL join:
// autos_imported LEFT JOIN categories ON (7 = categories.parent_id and
// autos_imported.value = categories.value )
// THIS WOULD PROBABLY BE SUFFICIENT TO JOIN STATIC DATA - GIVEN PARENT_ID AND VALUE
By adding the join above to the 10 fields I discussed for a 30 record page, would eliminate 300 calls.
These are my suggestions and needs.
I would be happy to help with this code if you would like as I have also chosen PN as my framework of choice.
Thanks so much,
--Joe -
- Rank: Team Member
- Registered: Jan 05, 2003
- Last visit: May 28, 2010
- Posts: 775
Posted: Jan 29, 2008 - 11:11 AM
Hi Joe,
reading your posts, I see what you're trying to do but I'm not sure I agree with the approach. There are a number of reasons for this:
1) I am opposed to changing something as deeply embedded in the system as DBUtil-auto-join this late in the game. This can of course be done, but I we're really close to RC3 now and I don't want to touch core functionality anymore.
2) The change you propose would require a change to the auto-join logic which I'm not sure I like. It's building a special case where there really should be generic code.
As such, I would propose the following compromise:
a) re-add the value field
b) write a plugin which gets the category based on the value field for a given parent ID. Since the parent ID is known, this can be done quite efficiently in a single SQL statement, the results of which can then be cached and re-used for sequential lookups.
Lastly a comment on your comment:
Quote
In response to your question about how I may be unable to use the built-in categorization services, I may be confused.
What I meant by this was that you can use categories, and you'll be able to use the value column as you wish, but you will have to store the value column mapping for your data in *your* tables (since the built in category mapping (as used but the Quotes and FAQ modules) uses category-IDs).
Greetings
R -
- Rank: Registered User
- Registered: Jun 18, 2005
- Last visit: Jan 29, 2008
- Posts: 9
Posted: Jan 29, 2008 - 12:08 PM
Hi Robert,
Great reply! I am no longer confused as your explanation on how I must store the value. This is exactly what I am using it for when dealing with static data values.
I can understand your need not to change much due to the upcomgin RC3 release. But, I know I must bring a valid point in regards to number of queries per page. I think its definately something to think about
. Therefore, this plugin you mention will be called for each static data value I need to lookup.
Thanks again and let me know when I can use/test,
--Joe -
- Rank: Registered User
- Registered: Jun 18, 2005
- Last visit: Jan 29, 2008
- Posts: 9
Posted: Jan 29, 2008 - 02:01 PM
FYI - I just got home and wanted to see what code may be affected for this join to happen. It appears the joinInfo array eventually gets past to the function DBUtil::_processJoinArray(). I think it may be easier than realized for me to modify this one function and perform my joins based on the object's defined joinInfo while calling the object's get() method. -
- Rank: Team Member
- Registered: Jan 05, 2003
- Last visit: May 28, 2010
- Posts: 775
Posted: Jan 30, 2008 - 09:52 AM
OK, after some discussion on the coredevs list the consensus is that it's too late to add this feature to the 0.8 codeline. We will examine this (or another appropriate alternate solution) for the 0.81 codeline.
In the meantime, I can help you implement this for your own purposes so that you can implement the features you need. We can work together to achieve a clean solution which is then hopefully suitable to include in the 0.81 branch.
I'm sorry if this is not the solution you were hoping for but this is the most reasonable course of action. You can contact me on rgasch]at[gmail.com to discuss the details of this.
Greetings
R
- Moderated by:
- Support
