Research
I do not have a lot of publications yet, but I hope that the ones I have right now are of interest for you. If you have any questions or are interested in commenting my work I will be happy to receive any comments.
1.
Font-Julian, Cristina I.; Orduña-Malea, Enrique; Codina, Lluís
ChatGPT Search as a tool for scholarly tasks: evolution or devolution? Journal Article
In: Infonomy, vol. 2, no. 5, pp. 15, 2024, ISSN: 2990-2290.
Abstract | Links | BibTeX | Tags: A-GEO, Academic Generative Engine Optimization, Academic Tasks, AI, ChatGPT, ChatGPT Search, Generative Artificial Intelligente, Information Retrieval, Link Analysis, Narrative Synthesis, Quantitative vs Qualitative, Scholarly Tasks, Search Engines, Web Search
@article{chatgpt-search,
title = {ChatGPT Search as a tool for scholarly tasks: evolution or devolution?},
author = {Cristina I. Font-Julian and Enrique Orduña-Malea and Lluís Codina},
doi = { https://doi.org/10.3145/infonomy.24.059},
issn = {2990-2290},
year = {2024},
date = {2024-11-26},
journal = {Infonomy},
volume = {2},
number = {5},
pages = {15},
abstract = {ChatGPT Search was launched on October 31 by OpenAI as a new AI-powered search engine. Among its features, it stands out for its ability to retrieve information from various online sources, including scholarly databases, which potentially allows the use of this tool for academic tasks, both quantitative and qualitative. To test its features, five academic tasks are designed: two quantitative (collecting hit count estimates from Google Search and scraping bibliometric indicators from ResearchGate); two qualitative tasks (performing a narrative synthesis of an academic topic and generating a brief academic author profile), and a mixed task (identifying, collecting and describing a list of publications from Google Scholar Profiles). The results show the inability of ChatGPT Search to conduct quantitative tasks correctly, fabricating the results (hallucination). Qualitative tasks are performed with better results; however, errors are detected, which prevent recommending the tool without manual analysis and refinement. Finally, the ability to generate links to scientific publications can open up competition among academic sites to be mentioned in the ChatGPT Search responses, giving rise to Academic Generative Engine Optimization (A-GEO).},
keywords = {A-GEO, Academic Generative Engine Optimization, Academic Tasks, AI, ChatGPT, ChatGPT Search, Generative Artificial Intelligente, Information Retrieval, Link Analysis, Narrative Synthesis, Quantitative vs Qualitative, Scholarly Tasks, Search Engines, Web Search},
pubstate = {published},
tppubtype = {article}
}
ChatGPT Search was launched on October 31 by OpenAI as a new AI-powered search engine. Among its features, it stands out for its ability to retrieve information from various online sources, including scholarly databases, which potentially allows the use of this tool for academic tasks, both quantitative and qualitative. To test its features, five academic tasks are designed: two quantitative (collecting hit count estimates from Google Search and scraping bibliometric indicators from ResearchGate); two qualitative tasks (performing a narrative synthesis of an academic topic and generating a brief academic author profile), and a mixed task (identifying, collecting and describing a list of publications from Google Scholar Profiles). The results show the inability of ChatGPT Search to conduct quantitative tasks correctly, fabricating the results (hallucination). Qualitative tasks are performed with better results; however, errors are detected, which prevent recommending the tool without manual analysis and refinement. Finally, the ability to generate links to scientific publications can open up competition among academic sites to be mentioned in the ChatGPT Search responses, giving rise to Academic Generative Engine Optimization (A-GEO).
2.
González-Llinares, Javier; Font-Julian, Cristina I.; Orduña-Malea, Enrique
Universidades en Google: hacia un modelo de análisis multinivel del posicionamiento web académico Journal Article
In: Revista Española De Documentación Científica, vol. 43, no. 2, pp. e260, 2020, ISBN: 1988-4621.
Abstract | Links | BibTeX | Tags: Academic SEO, Cybermetrics, Evaluation Model, Google, Search Engines, Search Engines Optimization, SEO, Universities, Web positioning, Web Visibility, Webometrics
@article{nokey,
title = {Universidades en Google: hacia un modelo de análisis multinivel del posicionamiento web académico},
author = {Javier González-Llinares and Cristina I. Font-Julian and Enrique Orduña-Malea},
doi = {10.3989/redc.2020.2.1691},
isbn = {1988-4621},
year = {2020},
date = {2020-04-17},
urldate = {2020-04-17},
journal = {Revista Española De Documentación Científica},
volume = {43},
number = {2},
pages = {e260},
abstract = {Se propone un modelo de análisis del posicionamiento web de universidades basado en un vocabulario de palabras clave categorizadas según las distintas misiones universitarias, que se aplica a una universidad (Universitat Politècnica de València) para comprobar su idoneidad. A partir de un vocabulario de 164 palabras clave se construyeron 290 consultas web que fueron ejecutadas en Google, recopilando los 20 primeros resultados obtenidos para cada consulta. Los resultados confirman que las universidades obtienen un posicionamiento web variable en función de la dimensión vinculada a la consulta web y que las páginas web vinculadas a la docencia (especialmente Grados) son las que mejor posicionan, incluso para consultas web orientadas a investigación. Con todo, se observa un posicionamiento bajo no sólo para la UPV sino para las universidades públicas presenciales españolas (sólo el 27% del total de resultados en el Top 20 corresponde a alguna de estas universidades). Se concluye que el análisis multinivel es necesario para estudiar el posicionamiento web de las universidades y que el modelo propuesto es viable y escalable. No obstante, se han identificado ciertas limitaciones (dependencia del vocabulario utilizado y alta variabilidad de datos) que deben tenerse en cuenta en el diseño de este tipo de modelos de análisis.},
keywords = {Academic SEO, Cybermetrics, Evaluation Model, Google, Search Engines, Search Engines Optimization, SEO, Universities, Web positioning, Web Visibility, Webometrics},
pubstate = {published},
tppubtype = {article}
}
Se propone un modelo de análisis del posicionamiento web de universidades basado en un vocabulario de palabras clave categorizadas según las distintas misiones universitarias, que se aplica a una universidad (Universitat Politècnica de València) para comprobar su idoneidad. A partir de un vocabulario de 164 palabras clave se construyeron 290 consultas web que fueron ejecutadas en Google, recopilando los 20 primeros resultados obtenidos para cada consulta. Los resultados confirman que las universidades obtienen un posicionamiento web variable en función de la dimensión vinculada a la consulta web y que las páginas web vinculadas a la docencia (especialmente Grados) son las que mejor posicionan, incluso para consultas web orientadas a investigación. Con todo, se observa un posicionamiento bajo no sólo para la UPV sino para las universidades públicas presenciales españolas (sólo el 27% del total de resultados en el Top 20 corresponde a alguna de estas universidades). Se concluye que el análisis multinivel es necesario para estudiar el posicionamiento web de las universidades y que el modelo propuesto es viable y escalable. No obstante, se han identificado ciertas limitaciones (dependencia del vocabulario utilizado y alta variabilidad de datos) que deben tenerse en cuenta en el diseño de este tipo de modelos de análisis.
3.
Font-Julian, Cristina I.; Orduña-Malea, Enrique; Ontalba-Ruipérez, José-Antonio
Hit count estimate variability for website-specific queries in search engines: The case for rare disease association websites Journal Article
In: Aslib Journal of Information Management, vol. 70, no. 2, pp. 192-213, 2018, ISSN: 2050-3806.
Abstract | Links | BibTeX | Tags: Bing, Google, Hit Count Estimates, Rare Diseases, Search Engines, Website Page Count
@article{nokey,
title = {Hit count estimate variability for website-specific queries in search engines: The case for rare disease association websites},
author = {Cristina I. Font-Julian and Enrique Orduña-Malea and José-Antonio Ontalba-Ruipérez},
doi = {10.1108/AJIM-10-2017-0226},
issn = {2050-3806},
year = {2018},
date = {2018-02-07},
urldate = {2018-02-07},
journal = {Aslib Journal of Information Management},
volume = {70},
number = {2},
pages = {192-213},
abstract = {Purpose
The purpose of this paper is to determine the effect of the chosen search engine results page (SERP) on the website-specific hit count estimation indicator.
Design/methodology/approach
A sample of 100 Spanish rare disease association websites is analysed, obtaining the website-specific hit count estimation for the first and last SERPs in two search engines (Google and Bing) at two different periods in time (2016 and 2017).
Findings
It has been empirically demonstrated that there are differences between the number of hits returned on the first and last SERP in both Google and Bing. These differences are significant when they exceed a threshold value on the first SERP.
Research limitations/implications
Future studies considering other samples, more SERPs and generating different queries other than website page count (<site>) would be desirable to draw more general conclusions on the nature of quantitative data provided by general search engines.
Practical implications
Selecting a wrong SERP to calculate some metrics (in this case, website-specific hit count estimation) might provide misleading results, comparisons and performance rankings. The empirical data suggest that the first SERP captures the differences between websites better because it has a greater discriminating power and is more appropriate for webometric longitudinal studies.
Social implications
The findings allow improving future quantitative webometric analyses based on website-specific hit count estimation metrics in general search engines.
Originality/value
The website-specific hit count estimation variability between SERPs has been empirically analysed, considering two different search engines (Google and Bing), a set of 100 websites focussed on a similar market (Spanish rare diseases associations), and two annual samples, making this study the most exhaustive on this issue to date.
},
keywords = {Bing, Google, Hit Count Estimates, Rare Diseases, Search Engines, Website Page Count},
pubstate = {published},
tppubtype = {article}
}
Purpose
The purpose of this paper is to determine the effect of the chosen search engine results page (SERP) on the website-specific hit count estimation indicator.
Design/methodology/approach
A sample of 100 Spanish rare disease association websites is analysed, obtaining the website-specific hit count estimation for the first and last SERPs in two search engines (Google and Bing) at two different periods in time (2016 and 2017).
Findings
It has been empirically demonstrated that there are differences between the number of hits returned on the first and last SERP in both Google and Bing. These differences are significant when they exceed a threshold value on the first SERP.
Research limitations/implications
Future studies considering other samples, more SERPs and generating different queries other than website page count (<site>) would be desirable to draw more general conclusions on the nature of quantitative data provided by general search engines.
Practical implications
Selecting a wrong SERP to calculate some metrics (in this case, website-specific hit count estimation) might provide misleading results, comparisons and performance rankings. The empirical data suggest that the first SERP captures the differences between websites better because it has a greater discriminating power and is more appropriate for webometric longitudinal studies.
Social implications
The findings allow improving future quantitative webometric analyses based on website-specific hit count estimation metrics in general search engines.
Originality/value
The website-specific hit count estimation variability between SERPs has been empirically analysed, considering two different search engines (Google and Bing), a set of 100 websites focussed on a similar market (Spanish rare diseases associations), and two annual samples, making this study the most exhaustive on this issue to date.
The purpose of this paper is to determine the effect of the chosen search engine results page (SERP) on the website-specific hit count estimation indicator.
Design/methodology/approach
A sample of 100 Spanish rare disease association websites is analysed, obtaining the website-specific hit count estimation for the first and last SERPs in two search engines (Google and Bing) at two different periods in time (2016 and 2017).
Findings
It has been empirically demonstrated that there are differences between the number of hits returned on the first and last SERP in both Google and Bing. These differences are significant when they exceed a threshold value on the first SERP.
Research limitations/implications
Future studies considering other samples, more SERPs and generating different queries other than website page count (<site>) would be desirable to draw more general conclusions on the nature of quantitative data provided by general search engines.
Practical implications
Selecting a wrong SERP to calculate some metrics (in this case, website-specific hit count estimation) might provide misleading results, comparisons and performance rankings. The empirical data suggest that the first SERP captures the differences between websites better because it has a greater discriminating power and is more appropriate for webometric longitudinal studies.
Social implications
The findings allow improving future quantitative webometric analyses based on website-specific hit count estimation metrics in general search engines.
Originality/value
The website-specific hit count estimation variability between SERPs has been empirically analysed, considering two different search engines (Google and Bing), a set of 100 websites focussed on a similar market (Spanish rare diseases associations), and two annual samples, making this study the most exhaustive on this issue to date.