Worpress-Plugin API: retrieving meta-data according the usage of different filters

[tarifa]

hello and good day dear friends on NeoWin, 

 

 

first of all:; i hope youre all right and everything goes well at your site;: 

 

I'm currently working on a parser to make a small preview on the newest plugins in wordpress. at the moment i think i work from a URL given  out of the range of these:  https://de.wordpress.org/plugins/browse/popular/ - let us say the first 30 to 40 URL-pages.

I'd like to retrieve only the title of the page and https://wordpress.org/plugins/wp-job-manager and https://wordpress.org/plugins/ninja-forms and - let us say the most popular wp-plugins: 

a little chunk of information (a bit of text)

The project: for a list of meta-data of popular wordpress-plugins (cf. https://de.wordpress.org/plugins/browse/popular/ and gathering the first 50 URLs - that are
50 plugins which are of interest! The challenge is: i want to fetch meta-data of all the existing plugins.

What i subsequently want to filter out after the fetch is - those plugins that have the newest timestamp
- that are updated (most) recently. It is all aobut acutality...

https://wordpress.org/plugins/wp-job-manager
https://wordpress.org/plugins/ninja-forms
https://wordpress.org/plugins/participants-database ....and so on and so forth.

see the source: https://wordpress.org/plugins/wp-job-manager/ we have the following set of meta-data for each wordpress-plugin:
 

Version: 1.9.5.12
installations: 10,000+    
WordPress Version: 5.0 or higher
Tested up to: 5.4 PHP  
Version: 5.6 or higher    
Tags 3 Tags: database member sign-up form volunteer
Last updated: 19 hours ago
plugin-ratings

but as i saw today: there we have a api that can be used for this purpose too: the Wordpress-Plugins-API 

cf: https://developer.wordpress.org/reference/functions/plugins_api/

Browse: Home / Reference / Functions / plugins_api()
plugins_api( string $action, array|object $args = array() )
Retrieves plugin installer pages from the WordPress.org Plugins API.

Description
It is possible for a plugin to override the Plugin API result with three filters. Assume this is for plugins, which can extend on the Plugin Info to offer more choices. This is very powerful and must be used with care when overriding the filters. The first filter, ‘plugins_api_args’, is for the args and gives the action as the second parameter. The hook for ‘plugins_api_args’ must ensure that an object is returned. The second filter, ‘plugins_api’, allows a plugin to override the WordPress.org Plugin Installation API entirely. If $action is ‘query_plugins’ or ‘plugin_information’, an object MUST be passed. If $action is ‘hot_tags’ or ‘hot_categories’, an array MUST be passed. Finally, the third filter, ‘plugins_api_result’, makes it possible to filter the response object or array, depending on the $action type.

Supported arguments per action

(object|array|WP_Error) Response object or array on success, WP_Error on failure. See the function reference article for more information on the make-up of possible return values depending on the value of $action.

 

+--------------------+---------------+--------------------+----------+----------------+
|                    |               |                    |          |                |
+--------------------+---------------+--------------------+----------+----------------+
| Argument Name      | query_plugins | plugin_information | hot_tags | hot_categories |
| $slug              | No            | Yes                | No       | No             |
| $per_page          | Yes           | No                 | No       | No             |
| $page              | Yes           | No                 | No       | No             |
| $number            | No            | No                 | Yes      | Yes            |
| $search            | Yes           | No                 | No       | No             |
| $tag               | Yes           | No                 | No       | No             |
| $author            | Yes           | No                 | No       | No             |
| $user              | Yes           | No                 | No       | No             |
| $browse            | Yes           | No                 | No       | No             |
| $locale            | Yes           | Yes                | No       | No             |
| $installed_plugins | Yes           | No                 | No       | No             |
| $is_ssl            | Yes           | Yes                | No       | No             |
| $fields            | Yes           | Yes                | No       | No             |
+--------------------+---------------+--------------------+----------+----------------+

well  - i guess i can use this API for the above mentioned aims too?

look forward to hear from you 

regards Tarifa 

 

cf: https://developer.wordpress.org/reference/functions/plugins_api/

see the reference list  :
 

$fields = array(
    'active_installs' => true,           // rounded int
    'added' => true,                     // date
    'author' => true,                    // a href html
    'author_block_count' => true,        // int
    'author_block_rating' => true,       // int
    'author_profile' => true,            // url
    'banners' => true,                   // array( [low], [high] )
    'compatibility' => false,            // empty array?
    'contributors' => true,              // array( array( [profile], [avatar], [display_name] )
    'description' => false,              // string
    'donate_link' => true,               // url
    'download_link' => true,             // url
    'downloaded' => false,               // int
    // 'group' => false,                 // n/a
    'homepage' => true,                  // url
    'icons' => false,                    // array( [1x] url, [2x] url )
    'last_updated' => true,              // datetime
    'name' => true,                      // string
    'num_ratings' => true,               // int
    'rating' => true,                    // int
    'ratings' => true,                   // array( [5..0] )
    'requires' => true,                  // version string
    'requires_php' => true,              // version string
    // 'reviews' => false,               // n/a, part of 'sections'
    'screenshots' => true,               // array( array( [src],  ) )
    'sections' => true,                  // array( [description], [installation], [changelog], [reviews], ...)
    'short_description' => false,        // string
    'slug' => true,                      // string
    'support_threads' => true,           // int
    'support_threads_resolved' => true,  // int
    'tags' => true,                      // array( )
    'tested' => true,                    // version string
    'version' => true,                   // version string
    'versions' => true,                  // array( [version] url )
);

 

is this an appropiate way to gather the meta-data!?

 

what do you say-!?

[tarifa]

after some more investigations i think that i am on the right path., 

The API is the proper way to get the metadata. However it seems to be not very fast. For 50 calls it might take 1-2 min.  I have the feeling that WP intentionally makes this call slow to discourage others from mining the plugin information. But anyway - i like this option very much.

others too: see what i have found

https://wordpress.stackexchange.com/questions/345095/using-custom-code-how-can-i-fetch-data-from-the-wordpress-plugin-repo/345361#345361

 

Using custom code, how can I fetch data from the WordPress plugin repo?

 

In the WordPress repo, on the main page for every plugin, it displays the date of the plugin's last update (screenshot). From my own WordPress site, is it possible to fetch this data with custom PHP and/or JavaScript code? I would like to fetch this information for each plugin that is installed on my site, then display it on the Dashboard-->Plugins page.
Perhaps there is some sort of API available which makes this data accessible for all plugins in the WordPress repo?
If possible, I might also want to fetch other data related to the plugin, e.g. 'WordPress version' and 'Tested up to.'


Yes, there is, and you can check it out here.

There are examples linked from that Codex page, and you might want to use version 1.2 (GET requests only) or 1.1 of the API where these versions both have the response format in JSON.

And actually, there's also plugins_api() which make things easy for you; however, you'd need to manually load the file where the function is defined (wp-admin/includes/plugin-install.php).

Example using plugins_api()

// You may comment this out IF you're sure the function exists.
require_once ABSPATH . 'wp-admin/includes/plugin-install.php';

$args = [
    'slug' => 'woocommerce',
];

$data = plugins_api( 'plugin_information', $args );
//var_dump( $data );

if ( $data && ! is_wp_error( $data ) ) {
    echo 'Latest version: ' . $data->version;
}
Example with manual HTTP requests
$args = [
    'slug' => 'woocommerce',
];

$url = 'http://api.wordpress.org/plugins/info/1.2/';
$url = add_query_arg( [
    'action'  => 'plugin_information', // first param for plugins_api()
    'request' => $args,                // second param for plugins_api()
], $url );

$res = wp_remote_get( $url );
if ( ! is_wp_error( $res ) ) {
    $data = json_decode( wp_remote_retrieve_body( $res ) );
    //var_dump( $data );
    echo 'Latest version: ' . $data->version;
}

Either way, if you want to exclude certain fields like reviews and read-me sections like "description" and "installation", you can use the fields argument like so:

$args = [
    'slug' => 'woocommerce',
    'fields' => [
        'sections' => false, // excludes all readme sections
        'reviews'  => false, // excludes all reviews
    ],
];

 

my project; 

I'd like to retrieve only the title of the page and https://wordpress.org/plugins/wp
job-manager
https://wordpress.org/plugins/ninja-forms

a little chunk of information (a bit of text)

The project: for a list of meta-data of popular wordpress-plugins 
(cf. https://de.wordpress.org/plugins/browse/popular/ and gathering the first 50 URLs - that are 50 plugins which are of interest! The challenge is: i want to fetch meta-data of all the existing plugins. 

 

What i subsequently want to filter out after the fetch is - those plugins that have the newest timestamp 

that are updated (most) recently. It is all aobut acutality...

 

https://wordpress.org/plugins/wp-job-manager
https://wordpress.org/plugins/ninja-forms
https://wordpress.org/plugins/participants-database ....and so on and so forth.

see the source: https://wordpress.org/plugins/wp-job-manager/ we have the following set of meta-data for each wordpress-plugin:

 

 

Version: 1.9.5.12 
installations: 10,000+    
WordPress Version: 5.0 or higher 
Tested up to: 5.4 PHP  
Version: 5.6 or higher    
Tags 3 Tags: database member sign-up form volunteer
Last updated: 19 hours ago
plugin-ratings

but as i saw today: there we have a api that can be used for this purpose too: 

cf: https://developer.wordpress.org/reference/functions/plugins_api/

The API is the proper way to get the metadata. However it is not very fast.  but anyway - i will go this way.

[tarifa]

hi there - good day dear friends, 

 

 

found a solution  - this i want to share with you... 

cf https://stackoverflow.com/questions/51990613/getting-a-list-of-all-plugins 

 

There is a plugin API which we can query for plugins like this: $api = plugins_api( 'query_plugins', $args );

developer.wordpress.org/reference/functions/plugins_api –

we can make use of

 

plugins_api('hot_tags'), array('page' => 90000000000, 'number' => 90000000000). 

 

It returns some plugins but not all. 

 

we can go this way:

 

$api = plugins_api( 'query_plugins', [ 'per_page' => -

Quote

 

Because getting all plugins at once will be too heavy for the server, it is a better idea to do it in steps
we could do as many plugins at once as the server can handle. For the example I use a safe 100 plugins at once.

Everytime the script runs, it increments the "page" number with 1. So the next time the script runs the next 100 plugins are retrieved. The contents of the existing plugins.json will be parsed. The new plugins will be added (or overwritten if the plugin already is present) to the existing data, before encoding and saving it again.
If the page number is past the last, no results will be returned. This way the script knows there are no more plugins next. It then resets the page to 1, so it starts over.

we can use the wp_options table to keep track of the pages, simply because it's the quickest way. It would be better to use some kind of filesystem caching. That will be easier to reset manually if needed.we can set a cronjob to execute the script every x minutes.  Now the plugins.json file will build up and grow step by step, every time it runs.

 

 

// get the current "page", or if the option not exists, set page to 1.
$page = get_option( 'plugins_page' ) ? (int)get_option( 'plugins_page' ) : 1;

// get the plugin objects
$plugins = plugins_api( 'query_plugins', [
    'per_page' => 100,
    'page'     => $page,
    'fields'   => [
       //.........
    ]
] );

// increment the page, or when no results, reset to 1.
update_option( 'plugins_page', count( $plugins ) > 0 ? ++ $page : 1 );

// build up the data array
$newData = [];
foreach ( $plugins as $plugin ) {
    foreach ( $plugin as $key => $p ) {
        if ( $p->name != null ) {
            $newData[ $p->name ] = [ 'slug' => $p->slug ];
        }
    }
}

// get plugin data already in file.
// The last argument (true) is important. It makes json objects into
// associative arrays so they can be merged with array_merge.
$existingData = json_decode( file_get_contents( 'plugins.json' ), true );

// merge existing data with new data
$pluginData = array_merge( $existingData, $newData );

file_put_contents( 'plugins.json', json_encode( $pluginData ) );

 

Getting a list of plugins:  This will not return ALL plugins but it will return the top rated ones:

$plugins = plugins_api('query_plugins', array(
    'per_page' => 100,
    'browse' => 'top-rated',
    'fields' =>
        array(
            'short_description' => false,
            'description' => false,
            'sections' => false,
            'tested' => false,
            'requires' => false,
            'rating' => false,
            'ratings' => false,
            'downloaded' => false,
            'downloadlink' => false,
            'last_updated' => false,
            'added' => false,
            'tags' => false,
            'compatibility' => false,
            'homepage' => false,
            'versions' => false,
            'donate_link' => false,
            'reviews' => false,
            'banners' => false,
            'icons' => false,
            'active_installs' => false,
            'group' => false,
            'contributors' => false
        )));

 

we can save the data as JSON

Since the data that we get is huge and it will be bad for performance,   we can try to get the name and the slug out of the array and then we write it in a JSON file:

$plugins_json = '{' . PHP_EOL;
// Get only the name and the slug
foreach ($plugins as $plugin) {
    foreach ($plugin as $key => $p) {
        if ($p->name != null) {
            // Let's beautify the JSON
            $plugins_json .= '  "'. $p->name . '": {' . PHP_EOL;
            $plugins_json .= '      "slug": "' . $p->slug . '"' . PHP_EOL;
            end($plugin);
            $plugins_json .= ($key !== key($plugin)) ? '    },' . PHP_EOL : '   }' . PHP_EOL;
        }
    }
}
$plugins_json .= '}';
file_put_contents('plugins.json', $plugins_json);

Now we have a slim JSON file with only the data that we need.

To keep updating the JSON file, we run that script to create a JSON file every 24 hours by setting up a Cron Job.

just wanted to share this with you

 

have a great day... 😉

[+Nik Louch Subscriber²]

Something is rotten in the state of Denmark...